-- Logs begin at Wed 2022-05-25 15:14:11 UTC, end at Wed 2022-05-25 16:29:11 UTC. -- May 25 15:52:50.471468 dv-cinder-843277-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... May 25 15:52:50.482860 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini May 25 15:52:50.484992 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed May 25 15:52:50 2022] *** May 25 15:52:50.484992 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 May 25 15:52:50.484992 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 May 25 15:52:50.484992 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: nodename: dv-cinder-843277-2-iscsi May 25 15:52:50.484992 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: machine: x86_64 May 25 15:52:50.484992 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: clock source: unix May 25 15:52:50.484992 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: pcre jit disabled May 25 15:52:50.484992 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: detected number of CPU cores: 4 May 25 15:52:50.484992 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: current working directory: / May 25 15:52:50.484992 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: detected binary path: /usr/bin/uwsgi-core May 25 15:52:50.484992 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: your processes number limit is 47680 May 25 15:52:50.484992 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: your memory page size is 4096 bytes May 25 15:52:50.484992 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: detected max file descriptor number: 2048 May 25 15:52:50.484992 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: lock engine: pthread robust mutexes May 25 15:52:50.485896 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: thunder lock: enabled May 25 15:52:50.485896 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 May 25 15:52:50.485896 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] May 25 15:52:50.529151 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: Python main interpreter initialized at 0x55f46c3ef9c0 May 25 15:52:50.529151 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: python threads support enabled May 25 15:52:50.529151 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: your server socket listen backlog is limited to 100 connections May 25 15:52:50.529151 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: your mercy for graceful operations on workers is 90 seconds May 25 15:52:50.529483 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: mapped 403077 bytes (393 KB) for 2 cores May 25 15:52:50.529543 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: *** Operational MODE: preforking *** May 25 15:52:50.529830 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: *** uWSGI is running in multiple interpreter mode *** May 25 15:52:50.529830 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: spawned uWSGI master process (pid: 80222) May 25 15:52:50.530023 dv-cinder-843277-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. May 25 15:52:50.531387 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: spawned uWSGI worker 1 (pid: 80232, cores: 1) May 25 15:52:50.531387 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: spawned uWSGI worker 2 (pid: 80233, cores: 1) May 25 15:52:50.531387 dv-cinder-843277-2-iscsi devstack@keystone.service[80222]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... May 25 15:52:56.026657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 15:52:56.026657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: from cryptography.utils import int_from_bytes May 25 15:52:56.027511 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 15:52:56.027511 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: from cryptography.utils import int_from_bytes May 25 15:52:56.051018 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 15:52:56.051018 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: from cryptography.utils import int_from_bytes May 25 15:52:56.051524 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead May 25 15:52:56.051524 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: from cryptography.utils import int_from_bytes May 25 15:52:56.075747 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 15:52:56.075949 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 15:52:56.076155 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] command line args: [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 15:52:56.076420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 15:52:56.076551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ================================================================================ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 15:52:56.076846 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] admin_token = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.077094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] config_dir = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.077258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.077437 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] config_source = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.077605 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] debug = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.077837 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.078188 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.078355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.078632 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.078853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.079105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.079308 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_config_append = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.079452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.079606 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_dir = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.079814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_file = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.080056 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.080298 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.080518 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.080784 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.080954 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.081088 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.081333 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.081579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.081863 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.082090 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.082318 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.082543 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.082785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.083028 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.083298 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.083787 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.85/identity {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.084000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] publish_errors = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.084219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.084452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.084630 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.084870 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.085100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.085328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] standard_threads = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.085555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.085778 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.085953 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.086127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_journal = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.086300 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_json = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.086471 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_stderr = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.086643 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] use_syslog = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.086867 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.087116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.087351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.087607 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.087863 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.088117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.088352 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.088597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.088831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.089424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.089668 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.089916 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.090148 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.090394 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.090629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.090873 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.091116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.091348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.091599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.091875 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.092124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.092358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.092609 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.092866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.093115 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.093361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.093588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.093815 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.094042 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.094273 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.094520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.094764 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.094991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.095219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.095463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.095706 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.095962 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.096210 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.096437 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.096672 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.096952 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.097130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.097377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.097605 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.097830 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.098063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.098180 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} May 25 15:52:56.098296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.098408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} May 25 15:52:56.098521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.098633 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] command line args: [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} May 25 15:52:56.098760 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.098873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} May 25 15:52:56.098985 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ================================================================================ {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} May 25 15:52:56.099096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.099214 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.099326 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] admin_token = **** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.099445 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.099558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] config_dir = [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.099676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.099789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.100007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] config_source = [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.100137 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.100260 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] debug = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.100374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.100487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.100626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.100746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.100938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.101059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.101173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.101285 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.101397 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.101515 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.101643 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.101756 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] list_limit = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.101756 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] log_config_append = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.101974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.102092 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.102216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.102329 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] log_dir = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.102442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.102555 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] log_file = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.102672 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.102793 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.102906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.103027 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.103147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.103260 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.103372 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.103490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.103624 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.103799 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.103958 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.104087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.104087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.104307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.104307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.104536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.104536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.104747 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.104858 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.104972 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.105092 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.105205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.105316 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.105438 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.105438 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.105657 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.105842 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.105958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.106078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.106190 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.106303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.106303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.106519 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.85/identity {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.106634 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.106745 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] publish_errors = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.106863 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.106983 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.107096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.107211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.107211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.107427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.107427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.107643 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.107643 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.107874 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.107987 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] standard_threads = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.108099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.108223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.108345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.108458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.108570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.108691 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.108810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.108922 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] use_journal = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.109042 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.109155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] use_json = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.109155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] use_stderr = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.109366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.109477 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] use_syslog = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.109588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.109711 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} May 25 15:52:56.109825 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.109936 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.110047 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.110143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.110248 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.110355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.110466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.110562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.110656 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.110656 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.110838 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.110941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.111069 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.111174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.111302 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.111423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.111544 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.111650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.111761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.111914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.112040 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.112156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.112283 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.112399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.external = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.112508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.112620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.112734 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.112837 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.112957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.113115 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.113265 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.113394 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.password = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.113510 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.113626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] auth.token = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.113738 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.113859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.113984 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.114099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.114212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.114315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.114428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.114550 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.114654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.114766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.114878 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.114991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.115109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.115230 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.115334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.115445 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.115558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.115679 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.115792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.115946 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.116063 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.116182 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.116285 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.116397 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.116509 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.116632 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.116746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.116859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.116972 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.117091 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.117211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.117333 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.117453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.117567 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.117683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.117797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.117911 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.118104 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.118220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.118332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.118446 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.118559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.118673 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.118799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.118914 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.119027 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.119140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.119253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.119365 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.119487 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.119601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.119715 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.119843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.119958 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.120070 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.120197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.120324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.120447 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.120562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.120675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.120789 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.120903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.121026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.121141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.121254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.121367 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.121480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.121593 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.121714 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.121835 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.121949 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.122070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.122183 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.122297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.122417 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.122536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.122651 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.122770 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.122886 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.123005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.123127 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.123241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.123354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.123466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.123579 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.123692 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.123812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.123972 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.124100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.124216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.124330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.124451 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.124564 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.124687 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.124810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.124926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.125040 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.125154 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.125267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.125381 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.125502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.125618 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.125732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.125845 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.125959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.126074 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.126196 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.126311 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.126538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.126651 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.126764 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.126885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.127007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.127122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.127235 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.127357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.127472 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.127596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.127783 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.127955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.128072 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.128187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.128303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.128428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.128550 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.128665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.128785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.128898 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.129024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.129148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.129264 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.129378 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.129492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.129605 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.129727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.129849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.129965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.130079 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.130200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.130314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.130428 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.130550 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.130665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.130779 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.130893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.131008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.131122 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.131312 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.131430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.131543 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.131657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.131770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.131898 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.132028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.132145 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.132258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.132374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.132487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.132601 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.132723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.132839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.132965 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.133086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.133207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.133322 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.133445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.133561 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.133665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.133761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.133857 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.133970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.133970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.134192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.134306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.134421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.134535 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.134648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] auth.external = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.134770 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.134885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.134998 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.135118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.135234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.135355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.135471 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.135584 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] auth.password = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.135698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.135812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] auth.token = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.135947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.136073 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.136189 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.136302 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.136416 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.136530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.136643 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.136765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.136880 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.137002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.137117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.137230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.137344 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.137465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.137587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.137701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] credential.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.137815 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.137933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.138048 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.138169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.138285 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.138398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.138512 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.138626 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.138744 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.138869 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.138985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.139098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.139212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.139212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.139522 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.139662 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.139780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.139951 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.140066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.140191 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.140307 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.140430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.140546 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.140659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.140773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.140887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.141000 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.141122 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.141236 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.141350 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.141465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.141579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.141691 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] federation.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.141815 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.141937 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.142053 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.142166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.142280 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.142394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.142515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.142629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.142744 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.142858 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.142973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.143086 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.143208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.143323 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.143437 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.143551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.143667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.143787 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.143935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.144053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.144167 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.144314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.144432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.144545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.144668 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.144783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.144899 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.145020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.145136 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] role.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.145249 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.145372 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] role.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.145494 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.145609 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.145722 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.145836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.145957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.146080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.146195 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.146310 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.146423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.146537 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.146659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.146774 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.146888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.147004 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.147117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.147230 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.147351 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.147466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.147579 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.147693 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.147806 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.147962 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.148072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.148171 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.148267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.148364 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.148460 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.148558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.148663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.148663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.148864 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.148961 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.149058 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.149154 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.149259 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.149358 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.149456 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.149456 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.149670 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.149670 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.149891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.150008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.150008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.150229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.150343 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.150343 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.150564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.150564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.150564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.150564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.150564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.150564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.151180 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.151295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.151408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.151522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.151634 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.151748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.151894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.152011 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.152124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.152239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.152352 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.152465 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.152591 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.152714 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.152828 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.152941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.153054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.153168 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.153356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.153480 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.153594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.153708 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.153822 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.153936 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.154057 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.154173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.154286 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.154401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.154514 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.154628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.154749 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.154865 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.154977 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.155090 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.155204 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.155327 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.155443 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.155556 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.155669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.155782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.155952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.156079 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.156194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.156308 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.156422 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.156536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.156648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.156776 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.156893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.157007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.157129 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.157253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.157367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.157482 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.157596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.157717 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.157832 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.157945 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.158059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.158172 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.158285 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.158407 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.158522 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.158636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.158753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.158867 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.158980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.159103 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.159218 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.159331 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.159445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.159559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.159679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.159804 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.159966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.160080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.160194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.160309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.160422 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.160549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.160666 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.160780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.160894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.160999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.161110 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.161232 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.161232 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80233) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 15:52:56.161448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.161448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.161448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.161448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.161448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.161967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.161967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.161967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.162276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.162431 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.162671 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.162916 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.163166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.163411 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.163645 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.163918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.164164 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] resource.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.164416 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.164669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.164959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.165213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.165467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.165704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.165953 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.166191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.166433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.166672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] role.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.166921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] role.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.167155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.167405 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.167654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.167932 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.168183 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.168431 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.168680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.168929 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.169177 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.169409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.169658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.169906 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.170156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.170403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.170795 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.171036 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.171289 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.171540 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.171789 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.172067 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.172304 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.172558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.172809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.173062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.173297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.173532 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.173765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.174015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.174271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.174521 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.174760 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.174998 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.175231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.175517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] token.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.175799 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.176078 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.176318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.176577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.176827 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.177066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.177311 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.177554 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.177804 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.178041 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.178281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.178528 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.178774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.179022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.179254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.179500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.179800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.180098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.connection = **** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.180346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.180597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.180837 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.181079 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.181317 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.181567 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.181806 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.182045 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.182290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.182774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.183016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.183264 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.183499 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.183745 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.183997 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.184236 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.184472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} May 25 15:52:56.184643 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80232) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} May 25 15:52:56.190825 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.191080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.191345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.199854 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.209490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.209724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.209954 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.220222 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.228442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.238667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.238900 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.279888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.281151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.331185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.331382 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.331493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.331572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.396846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.397053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.397262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.397456 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.397553 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.397702 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.397786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.397850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.397988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.398070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.398070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.398293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.421074 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.421280 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.421519 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.421699 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.421798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.421993 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.422119 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.422119 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.422340 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.422406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.422480 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.422554 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.422797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.423012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.433634 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.434175 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.446246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.446401 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.446545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.446627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.446690 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.446757 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.446880 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.446972 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.447130 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.447257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.447387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.447473 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.447552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80232) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.447630 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80233) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} May 25 15:52:56.465068 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.465283 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 15:52:56.465503 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.465650 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.465790 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 15:52:56.465873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 15:52:56.465994 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.466113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.466240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} May 25 15:52:56.466334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.466439 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.466527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.466725 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.466852 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.467010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.467221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.478121 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.478393 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.478657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.478935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.479437 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.479717 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.480010 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.480290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.486930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.488395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.490214 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.490490 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.490766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.491038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.491313 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.491544 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.491643 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.491935 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.492061 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.492191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.492296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.492461 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.492589 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.492707 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.492824 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.492973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.493099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.493390 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.493666 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.493934 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.494204 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.494458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.518899 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.520653 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.520780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.520911 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.522354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.522638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.526098 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.526375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.526647 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.527854 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.528167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.528455 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.533871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.534097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.535707 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.536011 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.538606 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.538999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.539280 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.539545 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.539852 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.540116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.540381 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.540504 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.540689 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.540876 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.541068 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.541371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.541626 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.541927 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.542163 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.542426 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.559004 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.559276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.560928 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.561198 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.563939 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.565873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.567630 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.567980 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.569595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.569885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.573019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.575275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.576585 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.578872 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.580335 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.580638 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.580918 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.581203 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.581476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.581775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.582664 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.582959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.583253 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.583534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.583842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.584127 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.600919 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.601196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.601456 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.601713 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.603349 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.603634 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.603921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.604182 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.609632 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.611295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.611568 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.612185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.613926 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.614199 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.614646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.614921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.615196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.615475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.615743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.617281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.617555 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.617835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.618109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.618381 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.629033 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.629287 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.629597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.629876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.630144 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.630420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.630695 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.631909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.632187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.632472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.632765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.633029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.633301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.633596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.648520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.651527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.651751 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.654799 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.655034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.656666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.658121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.658249 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.658538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.658812 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.659727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.661404 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.661693 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.661984 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.666144 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.667735 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.669391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.670956 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.671084 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.671214 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.671511 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.671785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.674317 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.674645 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.674981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.675282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.680825 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.681104 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.681486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.684408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.684698 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.685099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.689285 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80233) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.689561 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.689837 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.690108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.690375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.690642 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.690937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.691226 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.691473 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.691756 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.692032 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.692289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.692557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.692827 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80233) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.692944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80232) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} May 25 15:52:56.693231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.693504 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.693760 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.694030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.694299 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.694570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.694850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.695126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.695402 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.695681 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.695968 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.696230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.696493 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80232) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} May 25 15:52:56.731327 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f46c3ef9c0 pid: 80233 (default app) May 25 15:52:56.734574 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846e147-862d-4059-85b3-94c65106b89c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:52:56.734872 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846e147-862d-4059-85b3-94c65106b89c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:52:56.735150 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8846e147-862d-4059-85b3-94c65106b89c None None] PATH_INFO: `/v3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:52:56.735296 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55f46c3ef9c0 pid: 80232 (default app) May 25 15:52:56.739483 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1/1] 10.222.0.85 () {50 vars in 791 bytes} [Wed May 25 15:52:56 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:04.731695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a96a5-ac69-40c9-a24a-e88d5e6a5af3 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:04.732329 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a96a5-ac69-40c9-a24a-e88d5e6a5af3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:04.732329 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7a96a5-ac69-40c9-a24a-e88d5e6a5af3 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:04.736706 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1/2] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:04 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:04.743854 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be341c3-3d1d-424e-929e-01c2069ffe38 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:04.744740 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be341c3-3d1d-424e-929e-01c2069ffe38 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:04.745233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9be341c3-3d1d-424e-929e-01c2069ffe38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:04.816222 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: INFO dbcounter [None req-9be341c3-3d1d-424e-929e-01c2069ffe38 None None] Registered counter for database keystone May 25 15:53:04.841166 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG oslo_db.sqlalchemy.engines [None req-9be341c3-3d1d-424e-929e-01c2069ffe38 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80233) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 15:53:04.849222 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writer thread running {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 15:53:05.030375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG passlib.handlers.bcrypt [None req-9be341c3-3d1d-424e-929e-01c2069ffe38 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80233) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 25 15:53:05.030722 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG passlib.handlers.bcrypt [None req-9be341c3-3d1d-424e-929e-01c2069ffe38 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80233) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 25 15:53:05.086333 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9be341c3-3d1d-424e-929e-01c2069ffe38 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:05.087987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9be341c3-3d1d-424e-929e-01c2069ffe38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:05.218500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 2/3] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:04 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 476 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 25 15:53:05.405162 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a96a5-ac69-40c9-a24a-e88d5e6a5af3 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:05.480967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: INFO dbcounter [None req-fd7a96a5-ac69-40c9-a24a-e88d5e6a5af3 None None] Registered counter for database keystone May 25 15:53:05.502329 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG oslo_db.sqlalchemy.engines [None req-fd7a96a5-ac69-40c9-a24a-e88d5e6a5af3 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80232) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} May 25 15:53:05.513679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writer thread running {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} May 25 15:53:05.645939 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7a96a5-ac69-40c9-a24a-e88d5e6a5af3 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:05.651186 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb5b5ca-33a9-4c8c-a9e9-e86aea264d60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:05.651902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5b5ca-33a9-4c8c-a9e9-e86aea264d60 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:05.652199 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5b5ca-33a9-4c8c-a9e9-e86aea264d60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:05.652487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb5b5ca-33a9-4c8c-a9e9-e86aea264d60 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:05.660997 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5b5ca-33a9-4c8c-a9e9-e86aea264d60 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:05.717812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: WARNING keystone.common.rbac_enforcer.enforcer [None req-8bb5b5ca-33a9-4c8c-a9e9-e86aea264d60 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 25 15:53:05.718160 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb5b5ca-33a9-4c8c-a9e9-e86aea264d60 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:05.729205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 2/4] 10.222.0.85 () {58 vars in 1324 bytes} [Wed May 25 15:53:05 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:53:05.735711 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be341c3-3d1d-424e-929e-01c2069ffe38 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:05.751280 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9be341c3-3d1d-424e-929e-01c2069ffe38 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:05.757022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88351e7-3e56-40bc-942f-a35c877120be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:05.758164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88351e7-3e56-40bc-942f-a35c877120be admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:05.758460 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88351e7-3e56-40bc-942f-a35c877120be admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:05.758750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88351e7-3e56-40bc-942f-a35c877120be admin admin] PATH_INFO: `/v3/projects/12da861032034c60b092c703b1fe5e90` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:05.760450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88351e7-3e56-40bc-942f-a35c877120be admin admin] RBAC: Authorizing `identity:get_project(project_id=12da861032034c60b092c703b1fe5e90)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:05.922111 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: WARNING keystone.common.rbac_enforcer.enforcer [None req-d88351e7-3e56-40bc-942f-a35c877120be admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. May 25 15:53:05.922347 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88351e7-3e56-40bc-942f-a35c877120be admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:05.927245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 3/5] 10.222.0.85 () {56 vars in 1207 bytes} [Wed May 25 15:53:05 2022] GET /identity/v3/projects/12da861032034c60b092c703b1fe5e90 => generated 344 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:08.064241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99692e14-b400-4009-9701-ba2b31e8dbe3 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:08.065118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99692e14-b400-4009-9701-ba2b31e8dbe3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:08.065118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99692e14-b400-4009-9701-ba2b31e8dbe3 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:08.071983 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 3/6] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:08 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:08.078563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80de98a-a60d-477f-9789-6fec436dca1e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:08.079285 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80de98a-a60d-477f-9789-6fec436dca1e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:08.079517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80de98a-a60d-477f-9789-6fec436dca1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:08.101253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d80de98a-a60d-477f-9789-6fec436dca1e None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:08.102801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d80de98a-a60d-477f-9789-6fec436dca1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:08.111372 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 4/7] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:08 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) May 25 15:53:08.133913 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99692e14-b400-4009-9701-ba2b31e8dbe3 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:08.148481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99692e14-b400-4009-9701-ba2b31e8dbe3 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:08.154099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a874876d-dc31-475f-8719-0ba671be82f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:08.154761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874876d-dc31-475f-8719-0ba671be82f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:08.155053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874876d-dc31-475f-8719-0ba671be82f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:08.155336 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a874876d-dc31-475f-8719-0ba671be82f2 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:08.164846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874876d-dc31-475f-8719-0ba671be82f2 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:08.166682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a874876d-dc31-475f-8719-0ba671be82f2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:08.176912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 4/8] 10.222.0.85 () {58 vars in 1324 bytes} [Wed May 25 15:53:08 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 15:53:08.182765 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80de98a-a60d-477f-9789-6fec436dca1e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:08.192717 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80de98a-a60d-477f-9789-6fec436dca1e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:08.198492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a009454-0f87-4ecd-bae6-126da2727a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:08.199391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a009454-0f87-4ecd-bae6-126da2727a5c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:08.199685 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a009454-0f87-4ecd-bae6-126da2727a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:08.200001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a009454-0f87-4ecd-bae6-126da2727a5c admin admin] PATH_INFO: `/v3/users/7effefc613a44b799e8f81fc6a35b678` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:08.201558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a009454-0f87-4ecd-bae6-126da2727a5c admin admin] RBAC: Authorizing `identity:get_user(user_id=7effefc613a44b799e8f81fc6a35b678)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:08.203438 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a009454-0f87-4ecd-bae6-126da2727a5c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:08.205101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 5/9] 10.222.0.85 () {56 vars in 1198 bytes} [Wed May 25 15:53:08 2022] GET /identity/v3/users/7effefc613a44b799e8f81fc6a35b678 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:10.443067 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827078ec-20d0-4953-92f2-dcaeaa9c1508 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:10.443600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827078ec-20d0-4953-92f2-dcaeaa9c1508 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:10.443600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-827078ec-20d0-4953-92f2-dcaeaa9c1508 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:10.444997 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 5/10] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:10.449289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43f438-baa6-48da-9707-b758b665ed78 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:10.449710 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43f438-baa6-48da-9707-b758b665ed78 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:10.449918 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc43f438-baa6-48da-9707-b758b665ed78 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:10.450209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d51922-54e2-47c9-bb66-5580f5d05bbe None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:10.450650 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d51922-54e2-47c9-bb66-5580f5d05bbe None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:10.451081 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05d51922-54e2-47c9-bb66-5580f5d05bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:10.451937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 6/11] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:10.458660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a0e04-4a13-4138-8426-3b8ff400482e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:10.458956 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a0e04-4a13-4138-8426-3b8ff400482e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:10.459244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83a0e04-4a13-4138-8426-3b8ff400482e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:10.477859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d83a0e04-4a13-4138-8426-3b8ff400482e None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:10.478030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d83a0e04-4a13-4138-8426-3b8ff400482e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:10.493189 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG passlib.handlers.bcrypt [None req-05d51922-54e2-47c9-bb66-5580f5d05bbe None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80232) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} May 25 15:53:10.493620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG passlib.handlers.bcrypt [None req-05d51922-54e2-47c9-bb66-5580f5d05bbe None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80232) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} May 25 15:53:10.539661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-05d51922-54e2-47c9-bb66-5580f5d05bbe None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:10.541294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-05d51922-54e2-47c9-bb66-5580f5d05bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:10.562043 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 7/12] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:10.584795 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a0e04-4a13-4138-8426-3b8ff400482e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:10.610908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:10.642211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 6/13] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:10.663890 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d51922-54e2-47c9-bb66-5580f5d05bbe None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:10.691193 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83a0e04-4a13-4138-8426-3b8ff400482e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:10.724104 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92fee7-0407-4cf9-8ddb-619da1a471a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:10.724912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92fee7-0407-4cf9-8ddb-619da1a471a0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:10.725201 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92fee7-0407-4cf9-8ddb-619da1a471a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:10.725486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d92fee7-0407-4cf9-8ddb-619da1a471a0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:10.737991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d92fee7-0407-4cf9-8ddb-619da1a471a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:10.749620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-5d92fee7-0407-4cf9-8ddb-619da1a471a0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:10.749620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:10.749620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:10.749620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:10.749620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:10.749620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:10.749620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:10.749620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:10.749620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:10.749620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:10.749620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:10.749620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:10.749620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:10.749620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:10.749620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:10.749620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:10.750816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:10.750816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:10.750816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:10.750816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:10.750816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:10.750816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:10.750816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:10.750816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:10.750816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:10.750816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:10.750816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:10.750816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:53:10.750816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:10.750816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:10.750816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:10.750816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:10.752292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:10.752292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:10.752292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:10.752292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:10.752292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:10.752292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:10.752292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:10.752292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:10.752292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:10.752292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:10.752292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:10.752292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:53:10.752292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 8/14] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:53:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:10.756656 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92fee7-0407-4cf9-8ddb-619da1a471a0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:10.768099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05d51922-54e2-47c9-bb66-5580f5d05bbe None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:10.770922 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:10.805168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43628fde-cae1-4790-a3c2-3ff1eb683bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:10.806093 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43628fde-cae1-4790-a3c2-3ff1eb683bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:10.806388 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43628fde-cae1-4790-a3c2-3ff1eb683bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:10.806662 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43628fde-cae1-4790-a3c2-3ff1eb683bd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:10.837025 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:10.855759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d92fee7-0407-4cf9-8ddb-619da1a471a0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:10.875250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43628fde-cae1-4790-a3c2-3ff1eb683bd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:10.876843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43628fde-cae1-4790-a3c2-3ff1eb683bd4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:10.889665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b72d25-65ec-446a-b7c1-718b98257a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:10.890318 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b72d25-65ec-446a-b7c1-718b98257a47 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:10.890628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b72d25-65ec-446a-b7c1-718b98257a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:10.890905 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09b72d25-65ec-446a-b7c1-718b98257a47 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:10.891665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b72d25-65ec-446a-b7c1-718b98257a47 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:10.892008 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b72d25-65ec-446a-b7c1-718b98257a47 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:10.893494 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09b72d25-65ec-446a-b7c1-718b98257a47 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:10.905254 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 9/15] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:10.910909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b72d25-65ec-446a-b7c1-718b98257a47 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:10.973550 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 7/16] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 15:53:10.979223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43628fde-cae1-4790-a3c2-3ff1eb683bd4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:10.982347 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:11.009914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09b72d25-65ec-446a-b7c1-718b98257a47 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:11.042873 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9459bb-4e01-47df-ade9-e25aa829320a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:11.043758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9459bb-4e01-47df-ade9-e25aa829320a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:11.044079 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9459bb-4e01-47df-ade9-e25aa829320a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:11.044354 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9459bb-4e01-47df-ade9-e25aa829320a None admin] PATH_INFO: `/v3/users/7effefc613a44b799e8f81fc6a35b678` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:11.045588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9459bb-4e01-47df-ade9-e25aa829320a None admin] RBAC: Authorizing `identity:get_user(user_id=7effefc613a44b799e8f81fc6a35b678)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:11.047078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9459bb-4e01-47df-ade9-e25aa829320a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:11.048739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 10/17] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:10 2022] GET /identity/v3/users/7effefc613a44b799e8f81fc6a35b678 => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:11.054290 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9459bb-4e01-47df-ade9-e25aa829320a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:11.068539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:11.077229 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43628fde-cae1-4790-a3c2-3ff1eb683bd4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:11.110824 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6cd85-d550-463e-b3f5-f0d545dfd7de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:11.111866 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6cd85-d550-463e-b3f5-f0d545dfd7de None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:11.112348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6cd85-d550-463e-b3f5-f0d545dfd7de None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:11.112631 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae6cd85-d550-463e-b3f5-f0d545dfd7de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:11.131021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6cd85-d550-463e-b3f5-f0d545dfd7de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:11.132611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae6cd85-d550-463e-b3f5-f0d545dfd7de None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:11.153765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-9ae6cd85-d550-463e-b3f5-f0d545dfd7de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:53:11.153765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:11.153765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:11.153765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:11.153765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:11.153765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:11.153765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:11.153765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:11.153765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:11.153765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:11.153765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:11.153765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:11.153765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:53:11.153765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:53:11.153765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:53:11.154728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:53:11.154728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:11.154728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:11.154728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:11.154728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:11.154728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:11.154728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:11.154728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:11.154728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:11.154728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:11.154728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:11.154728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:11.154728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:11.154728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:11.154728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:11.154728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:11.154728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:11.155736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:53:11.155736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:53:11.155736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:53:11.155736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:11.156037 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9459bb-4e01-47df-ade9-e25aa829320a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:11.156119 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 8/18] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:53:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:53:11.160941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6cd85-d550-463e-b3f5-f0d545dfd7de None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:11.189938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7cf6b4-e800-4968-a584-f44c46690a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:11.190792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7cf6b4-e800-4968-a584-f44c46690a91 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:11.191085 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7cf6b4-e800-4968-a584-f44c46690a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:11.191410 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa7cf6b4-e800-4968-a584-f44c46690a91 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:11.192691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7cf6b4-e800-4968-a584-f44c46690a91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:11.194189 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa7cf6b4-e800-4968-a584-f44c46690a91 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:11.195841 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 11/19] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:53:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:11.201523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7cf6b4-e800-4968-a584-f44c46690a91 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:11.217815 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:11.287473 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:11.313115 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa7cf6b4-e800-4968-a584-f44c46690a91 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:11.351632 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a840f292-3b27-416d-97d0-81b55b76488d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:11.352358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840f292-3b27-416d-97d0-81b55b76488d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:11.352693 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840f292-3b27-416d-97d0-81b55b76488d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:11.352937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a840f292-3b27-416d-97d0-81b55b76488d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:11.353792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840f292-3b27-416d-97d0-81b55b76488d None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.domain.id=default, user.id=7effefc613a44b799e8f81fc6a35b678) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:11.354112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840f292-3b27-416d-97d0-81b55b76488d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:11.355597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a840f292-3b27-416d-97d0-81b55b76488d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:11.362453 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 12/20] 10.222.0.85 () {56 vars in 1322 bytes} [Wed May 25 15:53:11 2022] GET /identity/v3/role_assignments?user.id=7effefc613a44b799e8f81fc6a35b678&scope.domain.id=default&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 233 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:11.381596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae6cd85-d550-463e-b3f5-f0d545dfd7de None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:11.418862 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fd31e4-f7b5-4bda-95c6-8a8a1ea70d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:11.419557 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd31e4-f7b5-4bda-95c6-8a8a1ea70d88 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:11.419862 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd31e4-f7b5-4bda-95c6-8a8a1ea70d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:11.420155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fd31e4-f7b5-4bda-95c6-8a8a1ea70d88 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:11.420902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd31e4-f7b5-4bda-95c6-8a8a1ea70d88 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:11.421212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd31e4-f7b5-4bda-95c6-8a8a1ea70d88 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:11.422690 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fd31e4-f7b5-4bda-95c6-8a8a1ea70d88 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:11.442363 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 9/21] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:53:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:13.900774 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b007917-0333-42d2-a6c4-0692872fb451 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:13.901326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b007917-0333-42d2-a6c4-0692872fb451 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:13.901326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b007917-0333-42d2-a6c4-0692872fb451 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:13.902842 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 13/22] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:13.908420 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb3b75-aae9-408d-9f58-7d7202645479 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:13.909333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb3b75-aae9-408d-9f58-7d7202645479 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:13.909806 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2bb3b75-aae9-408d-9f58-7d7202645479 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:13.936826 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d2bb3b75-aae9-408d-9f58-7d7202645479 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:13.938332 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d2bb3b75-aae9-408d-9f58-7d7202645479 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:14.023278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 10/23] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:14.057650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b007917-0333-42d2-a6c4-0692872fb451 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:14.195607 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b007917-0333-42d2-a6c4-0692872fb451 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:14.218420 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97b6d4-f1ee-43ac-aa5c-c4d21fbbe3f0 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:14.219035 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97b6d4-f1ee-43ac-aa5c-c4d21fbbe3f0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:14.219505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f97b6d4-f1ee-43ac-aa5c-c4d21fbbe3f0 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:14.221161 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 11/24] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:53:14.230418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f2b07-f9ef-45a0-9975-877c52c8a2f2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:14.230980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f2b07-f9ef-45a0-9975-877c52c8a2f2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:14.231457 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e0f2b07-f9ef-45a0-9975-877c52c8a2f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:14.248206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26dec157-69f9-4bb3-8dc3-8b61fe0c059b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:14.249031 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dec157-69f9-4bb3-8dc3-8b61fe0c059b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:14.249320 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dec157-69f9-4bb3-8dc3-8b61fe0c059b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:14.249602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26dec157-69f9-4bb3-8dc3-8b61fe0c059b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:14.272709 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26dec157-69f9-4bb3-8dc3-8b61fe0c059b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:14.273219 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-2e0f2b07-f9ef-45a0-9975-877c52c8a2f2 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:14.274624 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2e0f2b07-f9ef-45a0-9975-877c52c8a2f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:14.288608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-26dec157-69f9-4bb3-8dc3-8b61fe0c059b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:14.288608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:14.288608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:14.288608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:14.288608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:14.288608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:14.288608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:14.288608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:14.288608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:14.288608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:14.288608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:14.288608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:14.288608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:14.288608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:14.288608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:14.288608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:14.289526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:14.289526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:14.289526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:14.289526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:14.289526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:14.289526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:14.289526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:14.289526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:14.289526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:14.289526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:14.289526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:14.289526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:53:14.289526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:14.289526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:14.289526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:14.289526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:14.290295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:14.290295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:14.290295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:14.290295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:14.290295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:14.290295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:14.290295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:14.290295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:14.290295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:14.290295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:14.290295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:14.290295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:53:14.291530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 14/25] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:53:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:14.306648 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bc975-4c45-46b3-89ad-bdf72ed82530 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:14.306944 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bc975-4c45-46b3-89ad-bdf72ed82530 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:14.307423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81bc975-4c45-46b3-89ad-bdf72ed82530 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:14.309614 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 15/26] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:53:14.310213 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bc975-4c45-46b3-89ad-bdf72ed82530 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:14.316524 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:14.365082 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:14.413757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 12/27] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:53:14.416623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a40b29-8dda-4d23-aa01-637e0758a522 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:14.417286 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a40b29-8dda-4d23-aa01-637e0758a522 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:14.419005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a40b29-8dda-4d23-aa01-637e0758a522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:14.447397 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-35a40b29-8dda-4d23-aa01-637e0758a522 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:14.454386 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-35a40b29-8dda-4d23-aa01-637e0758a522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:14.502192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81bc975-4c45-46b3-89ad-bdf72ed82530 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:14.564835 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60be71-5766-4c24-859f-705d6565a7ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:14.566158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60be71-5766-4c24-859f-705d6565a7ea None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:14.567265 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60be71-5766-4c24-859f-705d6565a7ea None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:14.567941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e60be71-5766-4c24-859f-705d6565a7ea None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:14.569996 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60be71-5766-4c24-859f-705d6565a7ea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:14.571234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60be71-5766-4c24-859f-705d6565a7ea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:14.573500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e60be71-5766-4c24-859f-705d6565a7ea None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:14.584983 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 13/28] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:14.586100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a40b29-8dda-4d23-aa01-637e0758a522 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:14.601123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 16/29] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:14.608273 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60be71-5766-4c24-859f-705d6565a7ea None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:14.766952 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:14.774222 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a40b29-8dda-4d23-aa01-637e0758a522 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:14.791300 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:14.836484 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e60be71-5766-4c24-859f-705d6565a7ea None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:14.840148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948da93-4d63-44e7-828c-b52069f4e736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:14.841204 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948da93-4d63-44e7-828c-b52069f4e736 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:14.841656 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948da93-4d63-44e7-828c-b52069f4e736 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:14.842113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a948da93-4d63-44e7-828c-b52069f4e736 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:14.853048 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948da93-4d63-44e7-828c-b52069f4e736 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:14.854765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a948da93-4d63-44e7-828c-b52069f4e736 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:14.872881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-a948da93-4d63-44e7-828c-b52069f4e736 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:53:14.872881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:14.872881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:14.872881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:14.872881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:14.872881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:14.872881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:14.872881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:14.872881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:14.872881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:14.872881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:14.872881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:14.872881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:53:14.872881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:53:14.872881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:53:14.873902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:53:14.873902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:14.873902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:14.873902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:14.873902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:14.873902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:14.873902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:14.873902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:14.873902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:14.873902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:14.873902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:14.873902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:14.873902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:14.873902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:14.873902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:14.873902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:14.873902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:14.874886 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:53:14.874886 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:53:14.874886 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:53:14.874886 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:14.876627 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 14/30] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:53:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:14.876810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6939ec2-53cf-4e85-8d05-23147a9fe48d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:14.877959 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6939ec2-53cf-4e85-8d05-23147a9fe48d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:14.878441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6939ec2-53cf-4e85-8d05-23147a9fe48d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:14.878930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6939ec2-53cf-4e85-8d05-23147a9fe48d None admin] PATH_INFO: `/v3/users/7effefc613a44b799e8f81fc6a35b678` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:14.880405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6939ec2-53cf-4e85-8d05-23147a9fe48d None admin] RBAC: Authorizing `identity:get_user(user_id=7effefc613a44b799e8f81fc6a35b678)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:14.882096 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948da93-4d63-44e7-828c-b52069f4e736 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:14.882367 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6939ec2-53cf-4e85-8d05-23147a9fe48d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:14.884285 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 17/31] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:14 2022] GET /identity/v3/users/7effefc613a44b799e8f81fc6a35b678 => generated 247 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:14.885932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4854c-6275-49df-890f-1149b93dc382 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:14.886464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4854c-6275-49df-890f-1149b93dc382 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:14.886942 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7a4854c-6275-49df-890f-1149b93dc382 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:14.888377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 18/32] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:14.889255 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4854c-6275-49df-890f-1149b93dc382 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:15.012876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7a4854c-6275-49df-890f-1149b93dc382 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:15.021934 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a948da93-4d63-44e7-828c-b52069f4e736 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:15.049530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e2886-84a1-48fb-8baf-9ecb2ffbdcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:15.049896 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e2886-84a1-48fb-8baf-9ecb2ffbdcf3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:15.050185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e2886-84a1-48fb-8baf-9ecb2ffbdcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:15.050464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-402e2886-84a1-48fb-8baf-9ecb2ffbdcf3 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:15.051217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e2886-84a1-48fb-8baf-9ecb2ffbdcf3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:15.051529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e2886-84a1-48fb-8baf-9ecb2ffbdcf3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:15.053120 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-402e2886-84a1-48fb-8baf-9ecb2ffbdcf3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:15.055357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:15.075904 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f63d2c8-68e6-45b9-ad21-67556310664b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:15.077017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63d2c8-68e6-45b9-ad21-67556310664b None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:15.077203 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 19/33] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:53:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:15.077675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63d2c8-68e6-45b9-ad21-67556310664b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:15.078147 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f63d2c8-68e6-45b9-ad21-67556310664b None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:15.079123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63d2c8-68e6-45b9-ad21-67556310664b None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:15.080875 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f63d2c8-68e6-45b9-ad21-67556310664b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:15.082948 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e2886-84a1-48fb-8baf-9ecb2ffbdcf3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:15.206426 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-402e2886-84a1-48fb-8baf-9ecb2ffbdcf3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:15.207458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 15/34] 10.222.0.85 () {60 vars in 1131 bytes} [Wed May 25 15:53:14 2022] POST /identity/v3/roles => generated 225 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:53:15.209162 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb06366-ce5e-48a4-9dad-19e84ac815c0 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:15.209462 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb06366-ce5e-48a4-9dad-19e84ac815c0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:15.209748 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb06366-ce5e-48a4-9dad-19e84ac815c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:15.249685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-0fb06366-ce5e-48a4-9dad-19e84ac815c0 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:15.254303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0fb06366-ce5e-48a4-9dad-19e84ac815c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:15.261845 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410b2a8-c109-4ba9-9714-a78a03482bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:15.263850 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410b2a8-c109-4ba9-9714-a78a03482bde None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:15.264371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410b2a8-c109-4ba9-9714-a78a03482bde None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:15.264867 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c410b2a8-c109-4ba9-9714-a78a03482bde None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:15.266934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c410b2a8-c109-4ba9-9714-a78a03482bde None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:15.269492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c410b2a8-c109-4ba9-9714-a78a03482bde None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:15.270500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:15.273086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 20/35] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:53:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:15.278032 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410b2a8-c109-4ba9-9714-a78a03482bde None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:15.364696 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 16/36] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:15.366894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:15.367381 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737aa57a-76b4-4e62-8a60-f0a866803754 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:15.369020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737aa57a-76b4-4e62-8a60-f0a866803754 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:15.369596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-737aa57a-76b4-4e62-8a60-f0a866803754 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:15.371095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 17/37] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:15.371956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737aa57a-76b4-4e62-8a60-f0a866803754 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:15.416587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c410b2a8-c109-4ba9-9714-a78a03482bde None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:15.459749 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5a8fb-6739-49c3-99a2-a20a4ec54c55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:15.460465 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5a8fb-6739-49c3-99a2-a20a4ec54c55 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:15.460758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5a8fb-6739-49c3-99a2-a20a4ec54c55 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:15.461059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74d5a8fb-6739-49c3-99a2-a20a4ec54c55 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:15.461790 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5a8fb-6739-49c3-99a2-a20a4ec54c55 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:15.463631 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74d5a8fb-6739-49c3-99a2-a20a4ec54c55 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:15.518371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-737aa57a-76b4-4e62-8a60-f0a866803754 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:15.554297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41f64e-58dd-4fb8-a77f-2db096527f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:15.555118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41f64e-58dd-4fb8-a77f-2db096527f5b None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:15.555420 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41f64e-58dd-4fb8-a77f-2db096527f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:15.555708 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41f64e-58dd-4fb8-a77f-2db096527f5b None admin] PATH_INFO: `/v3/domains/default/users/7effefc613a44b799e8f81fc6a35b678/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:15.558135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41f64e-58dd-4fb8-a77f-2db096527f5b None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=7effefc613a44b799e8f81fc6a35b678, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:15.559754 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca41f64e-58dd-4fb8-a77f-2db096527f5b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:15.575341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 18/38] 10.222.0.85 () {58 vars in 1361 bytes} [Wed May 25 15:53:15 2022] PUT /identity/v3/domains/default/users/7effefc613a44b799e8f81fc6a35b678/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:53:15.576758 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ce7fe-5bc0-453d-aaa3-dec69bf21ea2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:15.577055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ce7fe-5bc0-453d-aaa3-dec69bf21ea2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:15.577342 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0ce7fe-5bc0-453d-aaa3-dec69bf21ea2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:15.595454 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6a0ce7fe-5bc0-453d-aaa3-dec69bf21ea2 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:15.596756 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6a0ce7fe-5bc0-453d-aaa3-dec69bf21ea2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:15.599350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:15.615850 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 21/39] 10.222.0.85 () {60 vars in 1141 bytes} [Wed May 25 15:53:15 2022] POST /identity/v3/projects => generated 301 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:53:15.615850 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5a8fb-6739-49c3-99a2-a20a4ec54c55 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:15.695268 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 19/40] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:15.697070 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179b3a4-bb0d-4e03-b55c-334c049d0324 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:15.697479 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179b3a4-bb0d-4e03-b55c-334c049d0324 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:15.697835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2179b3a4-bb0d-4e03-b55c-334c049d0324 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:15.699406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 20/41] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:15.704954 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795166f-7aa4-427f-87f4-c7ad7647073f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:15.705360 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795166f-7aa4-427f-87f4-c7ad7647073f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:15.705864 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d795166f-7aa4-427f-87f4-c7ad7647073f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:15.725277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d795166f-7aa4-427f-87f4-c7ad7647073f None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:15.726692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d795166f-7aa4-427f-87f4-c7ad7647073f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:15.735046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74d5a8fb-6739-49c3-99a2-a20a4ec54c55 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:15.752019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:15.776440 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596c884-8143-405f-b24c-9d1681e23eea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:15.777168 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596c884-8143-405f-b24c-9d1681e23eea None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:15.777458 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596c884-8143-405f-b24c-9d1681e23eea None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:15.777747 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0596c884-8143-405f-b24c-9d1681e23eea None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:15.778537 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0596c884-8143-405f-b24c-9d1681e23eea None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:15.780200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0596c884-8143-405f-b24c-9d1681e23eea None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:15.802480 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 22/42] 10.222.0.85 () {60 vars in 1131 bytes} [Wed May 25 15:53:15 2022] POST /identity/v3/roles => generated 219 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:53:15.803530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596c884-8143-405f-b24c-9d1681e23eea None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:15.822719 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 21/43] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:53:15.854819 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795166f-7aa4-427f-87f4-c7ad7647073f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:15.935434 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:15.960436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0596c884-8143-405f-b24c-9d1681e23eea None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:15.985357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d795166f-7aa4-427f-87f4-c7ad7647073f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:16.015667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc7891-d566-4cfe-9620-71043fb1ca40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:16.016575 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc7891-d566-4cfe-9620-71043fb1ca40 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:16.016861 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc7891-d566-4cfe-9620-71043fb1ca40 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:16.017148 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ddc7891-d566-4cfe-9620-71043fb1ca40 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:16.018873 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc7891-d566-4cfe-9620-71043fb1ca40 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:16.020502 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ddc7891-d566-4cfe-9620-71043fb1ca40 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:16.022239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 23/44] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:53:15 2022] GET /identity/v3/domains/default => generated 199 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:16.027402 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08a76c88-9b12-4d97-a2a4-785700a7a473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:16.028178 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a76c88-9b12-4d97-a2a4-785700a7a473 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:16.028473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a76c88-9b12-4d97-a2a4-785700a7a473 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:16.028711 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc7891-d566-4cfe-9620-71043fb1ca40 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:16.028809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08a76c88-9b12-4d97-a2a4-785700a7a473 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:16.029701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a76c88-9b12-4d97-a2a4-785700a7a473 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:16.031134 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08a76c88-9b12-4d97-a2a4-785700a7a473 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:16.053069 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 22/45] 10.222.0.85 () {60 vars in 1131 bytes} [Wed May 25 15:53:15 2022] POST /identity/v3/roles => generated 223 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:53:16.086583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:16.162441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ddc7891-d566-4cfe-9620-71043fb1ca40 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:16.198398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860e8c15-ad56-443f-8f30-18418fed80da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:16.199148 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e8c15-ad56-443f-8f30-18418fed80da None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:16.199455 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e8c15-ad56-443f-8f30-18418fed80da None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:16.199781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860e8c15-ad56-443f-8f30-18418fed80da None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:16.200550 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e8c15-ad56-443f-8f30-18418fed80da None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:16.202055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-860e8c15-ad56-443f-8f30-18418fed80da None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:16.239414 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 24/46] 10.222.0.85 () {60 vars in 1141 bytes} [Wed May 25 15:53:16 2022] POST /identity/v3/projects => generated 312 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:53:17.894467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73432e6a-347a-4457-b6f6-d193be3fafbc None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:17.894988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73432e6a-347a-4457-b6f6-d193be3fafbc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:17.895069 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73432e6a-347a-4457-b6f6-d193be3fafbc None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:17.896563 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 23/47] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:17.901954 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcdf6e-c68d-4d13-87cc-ab199194d4a4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:17.902370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcdf6e-c68d-4d13-87cc-ab199194d4a4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:17.902513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8fcdf6e-c68d-4d13-87cc-ab199194d4a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:17.920474 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c8fcdf6e-c68d-4d13-87cc-ab199194d4a4 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:17.922018 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c8fcdf6e-c68d-4d13-87cc-ab199194d4a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:17.990975 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 25/48] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:18.013533 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73432e6a-347a-4457-b6f6-d193be3fafbc None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:18.118618 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73432e6a-347a-4457-b6f6-d193be3fafbc None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:18.145442 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:18.154086 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002f80f-c0a0-48cc-a38f-a463f6567d2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:18.154902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002f80f-c0a0-48cc-a38f-a463f6567d2d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:18.155198 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002f80f-c0a0-48cc-a38f-a463f6567d2d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:18.155485 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3002f80f-c0a0-48cc-a38f-a463f6567d2d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:18.166580 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3002f80f-c0a0-48cc-a38f-a463f6567d2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:18.175497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-3002f80f-c0a0-48cc-a38f-a463f6567d2d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:18.175497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:18.175497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:18.175497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:18.175497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:18.175497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:18.175497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:18.175497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:18.175497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:18.175497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:18.175497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:18.175497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:18.175497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:18.175497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:18.175497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:18.175497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:18.176398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:18.176398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:18.176398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:18.176398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:18.176398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:18.176398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:18.176398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:18.176398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:18.176398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:18.176398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:18.176398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:18.176398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:18.176398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:18.176398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:18.176398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:18.176398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:18.177173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:18.177173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:18.177173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:18.177173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:18.177173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:18.177173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:18.177173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:18.177173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:18.177173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:18.177173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:18.177173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:18.177173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:18.177173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 24/49] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:53:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:18.182720 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcdf6e-c68d-4d13-87cc-ab199194d4a4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:18.209831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:18.293734 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8fcdf6e-c68d-4d13-87cc-ab199194d4a4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:18.330980 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097fd95d-2535-4d63-9ce7-32296f35627d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:18.331658 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fd95d-2535-4d63-9ce7-32296f35627d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:18.331985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fd95d-2535-4d63-9ce7-32296f35627d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:18.332269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097fd95d-2535-4d63-9ce7-32296f35627d None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:18.333028 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fd95d-2535-4d63-9ce7-32296f35627d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:18.333341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fd95d-2535-4d63-9ce7-32296f35627d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:18.334753 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-097fd95d-2535-4d63-9ce7-32296f35627d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:18.343203 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 26/50] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:18 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:18.348994 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002f80f-c0a0-48cc-a38f-a463f6567d2d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:18.414003 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e24458-c253-4be7-89a1-07ae54b17bc7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:18.414225 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e24458-c253-4be7-89a1-07ae54b17bc7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:18.414526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45e24458-c253-4be7-89a1-07ae54b17bc7 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:18.415950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 27/51] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:18.421096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59b01b-7e8b-417a-ba26-2360b8d92d59 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:18.421399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59b01b-7e8b-417a-ba26-2360b8d92d59 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:18.421690 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d59b01b-7e8b-417a-ba26-2360b8d92d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:18.438388 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5d59b01b-7e8b-417a-ba26-2360b8d92d59 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:18.439610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5d59b01b-7e8b-417a-ba26-2360b8d92d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:18.459298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3002f80f-c0a0-48cc-a38f-a463f6567d2d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:18.497015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02eddd-178d-4611-a5ec-e6cb5b6e61bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:18.497915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02eddd-178d-4611-a5ec-e6cb5b6e61bb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:18.498201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02eddd-178d-4611-a5ec-e6cb5b6e61bb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:18.498484 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f02eddd-178d-4611-a5ec-e6cb5b6e61bb None admin] PATH_INFO: `/v3/users/7effefc613a44b799e8f81fc6a35b678` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:18.499779 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f02eddd-178d-4611-a5ec-e6cb5b6e61bb None admin] RBAC: Authorizing `identity:get_user(user_id=7effefc613a44b799e8f81fc6a35b678)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:18.501316 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f02eddd-178d-4611-a5ec-e6cb5b6e61bb None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:18.503022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 25/52] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:18 2022] GET /identity/v3/users/7effefc613a44b799e8f81fc6a35b678 => generated 247 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:18.508497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02eddd-178d-4611-a5ec-e6cb5b6e61bb None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:18.509601 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 28/53] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:53:18.532225 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59b01b-7e8b-417a-ba26-2360b8d92d59 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:18.548946 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:18.564526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:18.610603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f02eddd-178d-4611-a5ec-e6cb5b6e61bb None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:18.634072 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d59b01b-7e8b-417a-ba26-2360b8d92d59 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:18.643694 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f06f86-7e0d-4751-b962-681fa4bd55ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:18.644553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f06f86-7e0d-4751-b962-681fa4bd55ce None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:18.644851 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f06f86-7e0d-4751-b962-681fa4bd55ce None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:18.645141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f06f86-7e0d-4751-b962-681fa4bd55ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:18.646341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f06f86-7e0d-4751-b962-681fa4bd55ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:18.647848 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f06f86-7e0d-4751-b962-681fa4bd55ce None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:18.649588 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 26/54] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:53:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:18.655236 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f06f86-7e0d-4751-b962-681fa4bd55ce None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:18.668158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81f2af-cb13-4e76-a9f9-c0a5b3c9ee98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:18.668904 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81f2af-cb13-4e76-a9f9-c0a5b3c9ee98 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:18.669196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81f2af-cb13-4e76-a9f9-c0a5b3c9ee98 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:18.669487 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e81f2af-cb13-4e76-a9f9-c0a5b3c9ee98 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:18.670660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81f2af-cb13-4e76-a9f9-c0a5b3c9ee98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:18.672170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e81f2af-cb13-4e76-a9f9-c0a5b3c9ee98 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:18.673791 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 29/55] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:53:18 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:18.679268 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81f2af-cb13-4e76-a9f9-c0a5b3c9ee98 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:18.753191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f06f86-7e0d-4751-b962-681fa4bd55ce None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:18.766693 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:18.776888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:18.783606 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e81f2af-cb13-4e76-a9f9-c0a5b3c9ee98 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:18.788088 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d1554d4-6c30-46bc-8f2c-c70ec7fe01cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:18.788771 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1554d4-6c30-46bc-8f2c-c70ec7fe01cf None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:18.789070 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1554d4-6c30-46bc-8f2c-c70ec7fe01cf None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:18.789361 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1554d4-6c30-46bc-8f2c-c70ec7fe01cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:18.790213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1554d4-6c30-46bc-8f2c-c70ec7fe01cf None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.domain.id=default, user.id=7effefc613a44b799e8f81fc6a35b678) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:18.790538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1554d4-6c30-46bc-8f2c-c70ec7fe01cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:18.792054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d1554d4-6c30-46bc-8f2c-c70ec7fe01cf None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:18.800698 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 27/56] 10.222.0.85 () {56 vars in 1322 bytes} [Wed May 25 15:53:18 2022] GET /identity/v3/role_assignments?user.id=7effefc613a44b799e8f81fc6a35b678&scope.domain.id=default&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:18.820535 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e57b98d-d15e-43e2-8734-c062aa016790 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:18.821238 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57b98d-d15e-43e2-8734-c062aa016790 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:18.821530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57b98d-d15e-43e2-8734-c062aa016790 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:18.821812 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e57b98d-d15e-43e2-8734-c062aa016790 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:18.822560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57b98d-d15e-43e2-8734-c062aa016790 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:18.824100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e57b98d-d15e-43e2-8734-c062aa016790 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:18.853000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 30/57] 10.222.0.85 () {60 vars in 1140 bytes} [Wed May 25 15:53:18 2022] POST /identity/v3/projects => generated 298 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:53:21.039205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5e497-d6f9-4916-b59b-fcb91487ecde None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:21.039772 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5e497-d6f9-4916-b59b-fcb91487ecde None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:21.039772 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5e497-d6f9-4916-b59b-fcb91487ecde None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:21.041210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 28/58] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:21.046542 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eb408-c451-410c-8a1f-424eb8d13b8b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:21.046962 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eb408-c451-410c-8a1f-424eb8d13b8b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:21.047114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070eb408-c451-410c-8a1f-424eb8d13b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:21.066145 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-070eb408-c451-410c-8a1f-424eb8d13b8b None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:21.067355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-070eb408-c451-410c-8a1f-424eb8d13b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:21.136994 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 31/59] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:21.169674 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5e497-d6f9-4916-b59b-fcb91487ecde None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:21.280615 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5e497-d6f9-4916-b59b-fcb91487ecde None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:21.317459 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7a7f729-c202-4dd2-b2fe-4a3a17e77f9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:21.318276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7f729-c202-4dd2-b2fe-4a3a17e77f9b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:21.318576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7f729-c202-4dd2-b2fe-4a3a17e77f9b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:21.318859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7a7f729-c202-4dd2-b2fe-4a3a17e77f9b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:21.320105 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7f729-c202-4dd2-b2fe-4a3a17e77f9b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:21.321615 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7a7f729-c202-4dd2-b2fe-4a3a17e77f9b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:21.323255 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 29/60] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:53:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:21.329163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070eb408-c451-410c-8a1f-424eb8d13b8b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:21.394848 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:21.433273 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070eb408-c451-410c-8a1f-424eb8d13b8b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:21.468559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb2c531f-8e32-43c7-acd8-aa0d4bd48a46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:21.469229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c531f-8e32-43c7-acd8-aa0d4bd48a46 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:21.469512 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c531f-8e32-43c7-acd8-aa0d4bd48a46 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:21.469793 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb2c531f-8e32-43c7-acd8-aa0d4bd48a46 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:21.470505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c531f-8e32-43c7-acd8-aa0d4bd48a46 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:21.472033 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb2c531f-8e32-43c7-acd8-aa0d4bd48a46 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:21.500427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 32/61] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:53:21 2022] POST /identity/v3/users => generated 275 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:53:24.843675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263e7ae-75f6-4242-894f-80c150ccbeaf None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:24.844789 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263e7ae-75f6-4242-894f-80c150ccbeaf None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:24.845275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3263e7ae-75f6-4242-894f-80c150ccbeaf None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:24.846921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 30/62] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:24.852687 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e3c79-1c40-4710-9eda-a21fe6b7dbd9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:24.853540 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e3c79-1c40-4710-9eda-a21fe6b7dbd9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:24.854003 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3e3c79-1c40-4710-9eda-a21fe6b7dbd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:24.880901 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7e3e3c79-1c40-4710-9eda-a21fe6b7dbd9 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:24.882419 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7e3e3c79-1c40-4710-9eda-a21fe6b7dbd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:24.925341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1e5c9-0317-4654-8bbd-82842d6be1e2 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:24.925948 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1e5c9-0317-4654-8bbd-82842d6be1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:24.926408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee1e5c9-0317-4654-8bbd-82842d6be1e2 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:24.928007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 31/63] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:24.933383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7cd76-7ca8-4be9-a1b0-878d4428f4f6 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:24.933687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7cd76-7ca8-4be9-a1b0-878d4428f4f6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:24.933974 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a7cd76-7ca8-4be9-a1b0-878d4428f4f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:24.956921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-60a7cd76-7ca8-4be9-a1b0-878d4428f4f6 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:24.958418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-60a7cd76-7ca8-4be9-a1b0-878d4428f4f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:24.970868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 33/64] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:24.972530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506597d-1f68-4346-9cec-eb87bf5a92fc None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:24.973060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506597d-1f68-4346-9cec-eb87bf5a92fc None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:24.973540 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3506597d-1f68-4346-9cec-eb87bf5a92fc None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:24.975017 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 34/65] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:24.981490 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd268d-559e-428d-92f4-f436e2aba04a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:24.982038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd268d-559e-428d-92f4-f436e2aba04a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:24.982638 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abfd268d-559e-428d-92f4-f436e2aba04a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:25.005179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-abfd268d-559e-428d-92f4-f436e2aba04a None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:25.006381 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-abfd268d-559e-428d-92f4-f436e2aba04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:25.025464 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:25.044175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 32/66] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:53:25.045337 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7cd76-7ca8-4be9-a1b0-878d4428f4f6 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:25.075252 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:25.104613 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 35/67] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:25.105844 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd268d-559e-428d-92f4-f436e2aba04a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:25.165293 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a7cd76-7ca8-4be9-a1b0-878d4428f4f6 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:25.202562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934697ca-8f42-4384-863e-13e9d1666117 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:25.203802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934697ca-8f42-4384-863e-13e9d1666117 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:25.204352 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934697ca-8f42-4384-863e-13e9d1666117 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:25.204770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-934697ca-8f42-4384-863e-13e9d1666117 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:25.216356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-934697ca-8f42-4384-863e-13e9d1666117 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:25.221390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abfd268d-559e-428d-92f4-f436e2aba04a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:25.224952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-934697ca-8f42-4384-863e-13e9d1666117 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:25.224952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:25.224952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:25.224952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:25.224952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:25.224952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:25.224952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:25.224952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:25.224952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:25.224952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:25.224952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:25.224952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:25.224952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:25.224952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:25.224952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:25.224952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:25.225813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:25.225813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:25.225813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:25.225813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:25.225813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:25.225813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:25.225813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:25.225813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:25.225813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:25.225813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:25.225813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:25.225813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:25.225813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:25.225813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:25.225813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:25.225813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:25.226594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:25.226594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:25.226594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:25.226594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:25.226594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:25.226594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:25.226594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:25.226594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:25.226594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:25.226594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:25.226594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:25.226594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:25.228526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 33/68] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:25.229755 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934697ca-8f42-4384-863e-13e9d1666117 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:25.260887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9951f6-9812-42c9-881b-7701e7ce164e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:25.261725 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9951f6-9812-42c9-881b-7701e7ce164e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:25.262014 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9951f6-9812-42c9-881b-7701e7ce164e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:25.262293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9951f6-9812-42c9-881b-7701e7ce164e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:25.280667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9951f6-9812-42c9-881b-7701e7ce164e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:25.290948 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:25.293840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4c9951f6-9812-42c9-881b-7701e7ce164e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:53:25.293840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:25.293840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:25.293840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:25.293840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:25.293840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:25.293840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:25.293840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:25.293840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:25.293840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:25.293840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:25.293840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:25.293840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:25.293840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:25.293840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:25.293840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:25.294739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:25.294739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:25.294739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:25.294739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:25.294739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:25.294739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:25.294739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:25.294739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:25.294739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:25.294739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:25.294739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:25.294739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:53:25.294739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:25.294739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:25.294739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:25.294739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:25.295531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:25.295531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:25.295531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:25.295531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:25.295531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:25.295531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:25.295531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:25.295531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:25.295531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:25.295531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:25.295531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:53:25.295531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:53:25.303616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 36/69] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:53:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:25.303616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9951f6-9812-42c9-881b-7701e7ce164e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:25.356799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:25.385910 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-934697ca-8f42-4384-863e-13e9d1666117 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:25.418731 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9951f6-9812-42c9-881b-7701e7ce164e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:25.432317 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f98297-f23f-47f4-8cdc-cdd83a70b422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:25.433677 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f98297-f23f-47f4-8cdc-cdd83a70b422 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:25.434552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f98297-f23f-47f4-8cdc-cdd83a70b422 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:25.435140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f98297-f23f-47f4-8cdc-cdd83a70b422 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:25.450275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8f98297-f23f-47f4-8cdc-cdd83a70b422 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:25.457610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8bdd4-dd0d-4b4b-b05c-35e516526e9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:25.458167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-f8f98297-f23f-47f4-8cdc-cdd83a70b422 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:25.458167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:25.458167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:25.458167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:25.458167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:25.458167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:25.458167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:25.458167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:25.458167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:25.458167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:25.458167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:25.458167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:25.458167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:25.458167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:25.458167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:25.458167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:25.459007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:25.459007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:25.459007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:25.459007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:25.459007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:25.459007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:25.459007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:25.459007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:25.459007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:25.459007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:25.459007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:25.459007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:25.459007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:25.459007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:25.459007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:25.459007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:25.459777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:25.459777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:25.459777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:25.459777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:25.459777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:25.459777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:25.459777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:25.459777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:25.459777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:25.459777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:25.459777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:25.459777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:25.459777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 34/70] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:53:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:25.461912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8bdd4-dd0d-4b4b-b05c-35e516526e9e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:25.462734 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8bdd4-dd0d-4b4b-b05c-35e516526e9e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:25.463203 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca8bdd4-dd0d-4b4b-b05c-35e516526e9e None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:25.464053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f98297-f23f-47f4-8cdc-cdd83a70b422 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:25.464417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8bdd4-dd0d-4b4b-b05c-35e516526e9e None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:25.464919 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8bdd4-dd0d-4b4b-b05c-35e516526e9e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:25.466883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca8bdd4-dd0d-4b4b-b05c-35e516526e9e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:25.477353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 37/71] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:25.478615 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8bdd4-dd0d-4b4b-b05c-35e516526e9e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:25.567699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8f98297-f23f-47f4-8cdc-cdd83a70b422 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:25.574472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:25.576712 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca8bdd4-dd0d-4b4b-b05c-35e516526e9e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:25.579713 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:25.601902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45344e9e-b59b-4046-a367-cdab1e932f99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:25.602591 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45344e9e-b59b-4046-a367-cdab1e932f99 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:25.602886 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45344e9e-b59b-4046-a367-cdab1e932f99 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:25.603180 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45344e9e-b59b-4046-a367-cdab1e932f99 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:25.603972 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45344e9e-b59b-4046-a367-cdab1e932f99 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:25.604323 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45344e9e-b59b-4046-a367-cdab1e932f99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:25.605744 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45344e9e-b59b-4046-a367-cdab1e932f99 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:25.610576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2f2432-a997-4178-8f5d-23b35ef85d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:25.611234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f2432-a997-4178-8f5d-23b35ef85d6f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:25.611529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f2432-a997-4178-8f5d-23b35ef85d6f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:25.611833 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2f2432-a997-4178-8f5d-23b35ef85d6f None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:25.612601 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f2432-a997-4178-8f5d-23b35ef85d6f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:25.612922 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f2432-a997-4178-8f5d-23b35ef85d6f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:25.614343 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2f2432-a997-4178-8f5d-23b35ef85d6f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:25.617036 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 35/72] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:53:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:25.618106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d95177-7858-49e8-88f3-79f47fa8f350 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:25.618408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d95177-7858-49e8-88f3-79f47fa8f350 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:25.618695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d95177-7858-49e8-88f3-79f47fa8f350 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:25.620128 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 36/73] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:25.627332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 38/74] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:25 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:25.627332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b0dc3-9cb0-4b35-99c7-7f211814843d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:25.627332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b0dc3-9cb0-4b35-99c7-7f211814843d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:25.627627 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d95177-7858-49e8-88f3-79f47fa8f350 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:25.627722 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975b0dc3-9cb0-4b35-99c7-7f211814843d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:25.627722 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 39/75] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:25.627722 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b0dc3-9cb0-4b35-99c7-7f211814843d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:25.729598 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975b0dc3-9cb0-4b35-99c7-7f211814843d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:25.729800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d95177-7858-49e8-88f3-79f47fa8f350 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:25.761958 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dc554de-9ae2-40e9-a75d-5b92a6e805f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:25.762839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc554de-9ae2-40e9-a75d-5b92a6e805f2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:25.763130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc554de-9ae2-40e9-a75d-5b92a6e805f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:25.763417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dc554de-9ae2-40e9-a75d-5b92a6e805f2 None admin] PATH_INFO: `/v3/users/b9df4b9d4b0942339277d976457b2124` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:25.764661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7f979-53f3-48a4-9789-1e134b421285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:25.765562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7f979-53f3-48a4-9789-1e134b421285 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:25.765865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7f979-53f3-48a4-9789-1e134b421285 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:25.766152 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd7f979-53f3-48a4-9789-1e134b421285 None admin] PATH_INFO: `/v3/users/b9df4b9d4b0942339277d976457b2124` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:25.776893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc554de-9ae2-40e9-a75d-5b92a6e805f2 None admin] RBAC: Authorizing `identity:get_user(user_id=b9df4b9d4b0942339277d976457b2124)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:25.778708 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dc554de-9ae2-40e9-a75d-5b92a6e805f2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:25.785160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 40/76] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:25 2022] GET /identity/v3/users/b9df4b9d4b0942339277d976457b2124 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:25.785301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7f979-53f3-48a4-9789-1e134b421285 None admin] RBAC: Authorizing `identity:get_user(user_id=b9df4b9d4b0942339277d976457b2124)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:25.785389 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5769401-76f7-4b0e-9c93-28446f96066f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:25.785389 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5769401-76f7-4b0e-9c93-28446f96066f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:25.785389 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5769401-76f7-4b0e-9c93-28446f96066f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:25.785692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd7f979-53f3-48a4-9789-1e134b421285 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:25.785692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 37/77] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:25 2022] GET /identity/v3/users/b9df4b9d4b0942339277d976457b2124 => generated 275 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:25.785692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7f979-53f3-48a4-9789-1e134b421285 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:25.802786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c5769401-76f7-4b0e-9c93-28446f96066f None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:25.804043 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c5769401-76f7-4b0e-9c93-28446f96066f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:25.815966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:25.822046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:25.872009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 41/78] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:25.881189 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6533a5-019c-434c-969b-e812a199e636 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:25.881571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6533a5-019c-434c-969b-e812a199e636 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:25.881860 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae6533a5-019c-434c-969b-e812a199e636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:25.887897 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd7f979-53f3-48a4-9789-1e134b421285 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:25.899395 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ae6533a5-019c-434c-969b-e812a199e636 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:25.900620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ae6533a5-019c-434c-969b-e812a199e636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:25.922053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4163b7-9035-416d-b281-c1539ce85308 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:25.922941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4163b7-9035-416d-b281-c1539ce85308 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:25.923218 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4163b7-9035-416d-b281-c1539ce85308 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:25.923510 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4163b7-9035-416d-b281-c1539ce85308 None admin] PATH_INFO: `/v3/users/7effefc613a44b799e8f81fc6a35b678` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:25.924859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4163b7-9035-416d-b281-c1539ce85308 None admin] RBAC: Authorizing `identity:get_user(user_id=7effefc613a44b799e8f81fc6a35b678)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:25.926330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc4163b7-9035-416d-b281-c1539ce85308 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:25.928131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 38/79] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:25 2022] GET /identity/v3/users/7effefc613a44b799e8f81fc6a35b678 => generated 247 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:25.928708 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4163b7-9035-416d-b281-c1539ce85308 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:25.973002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 42/80] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:25.973856 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6533a5-019c-434c-969b-e812a199e636 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:26.030694 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:26.038215 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc4163b7-9035-416d-b281-c1539ce85308 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:26.060997 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:26.073694 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f941ec-d2fb-4a53-805c-25c103507985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:26.074366 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f941ec-d2fb-4a53-805c-25c103507985 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:26.074660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f941ec-d2fb-4a53-805c-25c103507985 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:26.074946 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78f941ec-d2fb-4a53-805c-25c103507985 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:26.076790 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae6533a5-019c-434c-969b-e812a199e636 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:26.111011 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7c108-4515-4809-9a72-0e0e75d9b562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:26.111739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7c108-4515-4809-9a72-0e0e75d9b562 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:26.112059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7c108-4515-4809-9a72-0e0e75d9b562 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:26.112343 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f7c108-4515-4809-9a72-0e0e75d9b562 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:26.141618 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f941ec-d2fb-4a53-805c-25c103507985 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:26.143137 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78f941ec-d2fb-4a53-805c-25c103507985 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:26.178638 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7c108-4515-4809-9a72-0e0e75d9b562 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:26.180179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f7c108-4515-4809-9a72-0e0e75d9b562 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:26.242064 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:26.250034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 39/81] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:53:26.250867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f941ec-d2fb-4a53-805c-25c103507985 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:26.262763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:26.279685 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 43/82] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:53:26.280543 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7c108-4515-4809-9a72-0e0e75d9b562 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:26.356209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78f941ec-d2fb-4a53-805c-25c103507985 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:26.379369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f7c108-4515-4809-9a72-0e0e75d9b562 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:26.389730 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9820b73-17db-4483-8c76-3366781187f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:26.390568 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9820b73-17db-4483-8c76-3366781187f3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:26.390860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9820b73-17db-4483-8c76-3366781187f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:26.391141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9820b73-17db-4483-8c76-3366781187f3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:26.392418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9820b73-17db-4483-8c76-3366781187f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:26.393843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9820b73-17db-4483-8c76-3366781187f3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:26.395467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 40/83] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:53:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:26.396252 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9820b73-17db-4483-8c76-3366781187f3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:26.418238 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1799ac4-105c-40dd-9e4c-e52b5e642dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:26.418901 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1799ac4-105c-40dd-9e4c-e52b5e642dfe None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:26.419195 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1799ac4-105c-40dd-9e4c-e52b5e642dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:26.419475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1799ac4-105c-40dd-9e4c-e52b5e642dfe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:26.456322 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:26.471769 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:26.489591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1799ac4-105c-40dd-9e4c-e52b5e642dfe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:26.491131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1799ac4-105c-40dd-9e4c-e52b5e642dfe None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:26.495977 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9820b73-17db-4483-8c76-3366781187f3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:26.530071 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18a65b-b9f1-4786-ba00-5db1e8628e06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:26.530883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18a65b-b9f1-4786-ba00-5db1e8628e06 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:26.531173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18a65b-b9f1-4786-ba00-5db1e8628e06 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:26.531624 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e18a65b-b9f1-4786-ba00-5db1e8628e06 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:26.541685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e18a65b-b9f1-4786-ba00-5db1e8628e06 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:26.549566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-4e18a65b-b9f1-4786-ba00-5db1e8628e06 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:26.549566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:26.549566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:26.549566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:26.549566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:26.549566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:26.549566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:26.549566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:26.549566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:26.549566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:26.549566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:26.549566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:26.549566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:26.549566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:26.549566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:26.549566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:26.550472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:26.550472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:26.550472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:26.550472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:26.550472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:26.550472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:26.550472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:26.550472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:26.550472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:26.550472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:26.550472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:26.550472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:26.550472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:26.550472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:26.550472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:26.550472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:26.556385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:26.556385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:26.556385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:26.556385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:26.556385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:26.556385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:26.556385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:26.556385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:26.556385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:26.556385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:26.556385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:26.556385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:26.556385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 41/84] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:26.556385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18a65b-b9f1-4786-ba00-5db1e8628e06 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:26.592078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 44/85] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:53:26.592937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1799ac4-105c-40dd-9e4c-e52b5e642dfe None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:26.654195 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e18a65b-b9f1-4786-ba00-5db1e8628e06 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:26.674734 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:26.677364 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:26.689451 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84155a50-17ef-4bc5-ab96-3c1c5d3f0927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:26.690332 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84155a50-17ef-4bc5-ab96-3c1c5d3f0927 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:26.690626 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84155a50-17ef-4bc5-ab96-3c1c5d3f0927 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:26.690913 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84155a50-17ef-4bc5-ab96-3c1c5d3f0927 None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:26.691001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1799ac4-105c-40dd-9e4c-e52b5e642dfe None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:26.692183 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84155a50-17ef-4bc5-ab96-3c1c5d3f0927 None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:26.693676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84155a50-17ef-4bc5-ab96-3c1c5d3f0927 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:26.695399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 42/86] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:26 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:26.696189 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84155a50-17ef-4bc5-ab96-3c1c5d3f0927 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:26.728089 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1c169-f7ed-4781-a897-a289939746cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:26.728975 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1c169-f7ed-4781-a897-a289939746cc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:26.729942 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1c169-f7ed-4781-a897-a289939746cc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:26.729942 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b1c169-f7ed-4781-a897-a289939746cc None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:26.730876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1c169-f7ed-4781-a897-a289939746cc None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:26.732400 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b1c169-f7ed-4781-a897-a289939746cc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:26.734119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 45/87] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:26 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:26.734894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1c169-f7ed-4781-a897-a289939746cc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:26.792607 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84155a50-17ef-4bc5-ab96-3c1c5d3f0927 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:26.826070 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b187a0-0185-493c-9317-cb926db752dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:26.826754 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b187a0-0185-493c-9317-cb926db752dd None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:26.827074 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b187a0-0185-493c-9317-cb926db752dd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:26.827353 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b187a0-0185-493c-9317-cb926db752dd None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:26.828108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b187a0-0185-493c-9317-cb926db752dd None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:26.829576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b187a0-0185-493c-9317-cb926db752dd None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:26.835976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b1c169-f7ed-4781-a897-a289939746cc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:26.856115 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 43/88] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:53:26 2022] POST /identity/v3/users => generated 289 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:53:26.856954 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b187a0-0185-493c-9317-cb926db752dd None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:26.871433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620ed6c-eba5-4868-9fc0-7643347a5588 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:26.872179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620ed6c-eba5-4868-9fc0-7643347a5588 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:26.872748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620ed6c-eba5-4868-9fc0-7643347a5588 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:26.872748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e620ed6c-eba5-4868-9fc0-7643347a5588 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:26.873528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620ed6c-eba5-4868-9fc0-7643347a5588 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:26.873851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620ed6c-eba5-4868-9fc0-7643347a5588 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:26.875329 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e620ed6c-eba5-4868-9fc0-7643347a5588 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:26.889763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 46/89] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:26.889763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620ed6c-eba5-4868-9fc0-7643347a5588 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:26.904101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:26.927649 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:26.993907 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b187a0-0185-493c-9317-cb926db752dd None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:27.001477 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e620ed6c-eba5-4868-9fc0-7643347a5588 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:27.038926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328cec4-417a-4d81-8f88-a0e650d3d8c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:27.039705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328cec4-417a-4d81-8f88-a0e650d3d8c0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:27.040020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328cec4-417a-4d81-8f88-a0e650d3d8c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:27.040314 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8328cec4-417a-4d81-8f88-a0e650d3d8c0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:27.041176 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328cec4-417a-4d81-8f88-a0e650d3d8c0 None admin] RBAC: Adding query filter params (role.id=1cc9b5d798c648c986537d006320581f, scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d, user.id=b9df4b9d4b0942339277d976457b2124) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:27.041505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328cec4-417a-4d81-8f88-a0e650d3d8c0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:27.041821 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2d313-4fc2-4870-8157-e9683dc2cfca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:27.042930 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2d313-4fc2-4870-8157-e9683dc2cfca None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:27.043052 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8328cec4-417a-4d81-8f88-a0e650d3d8c0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:27.043230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2d313-4fc2-4870-8157-e9683dc2cfca None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:27.043577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b2d313-4fc2-4870-8157-e9683dc2cfca None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:27.044939 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2d313-4fc2-4870-8157-e9683dc2cfca None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:27.046423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b2d313-4fc2-4870-8157-e9683dc2cfca None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:27.048309 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 44/90] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:26 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:27.049856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2d313-4fc2-4870-8157-e9683dc2cfca None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:27.054297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 47/91] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:26 2022] GET /identity/v3/role_assignments?user.id=b9df4b9d4b0942339277d976457b2124&scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d&role.id=1cc9b5d798c648c986537d006320581f => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:27.055114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328cec4-417a-4d81-8f88-a0e650d3d8c0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:27.159398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:27.199237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8328cec4-417a-4d81-8f88-a0e650d3d8c0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:27.207898 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b2d313-4fc2-4870-8157-e9683dc2cfca None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:27.230406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:27.258329 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3814a0-ba91-4a71-9510-7ce1cbfd08e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:27.259697 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3814a0-ba91-4a71-9510-7ce1cbfd08e5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:27.259697 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3814a0-ba91-4a71-9510-7ce1cbfd08e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:27.259925 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e3814a0-ba91-4a71-9510-7ce1cbfd08e5 None admin] PATH_INFO: `/v3/users/b9df4b9d4b0942339277d976457b2124` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:27.263326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3814a0-ba91-4a71-9510-7ce1cbfd08e5 None admin] RBAC: Authorizing `identity:get_user(user_id=b9df4b9d4b0942339277d976457b2124)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:27.264883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e3814a0-ba91-4a71-9510-7ce1cbfd08e5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:27.266560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 48/92] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:27 2022] GET /identity/v3/users/b9df4b9d4b0942339277d976457b2124 => generated 275 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:27.267401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3814a0-ba91-4a71-9510-7ce1cbfd08e5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:27.271971 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb6c29-aa12-4be3-b477-ffee3ef86aa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:27.272677 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb6c29-aa12-4be3-b477-ffee3ef86aa6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:27.272972 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb6c29-aa12-4be3-b477-ffee3ef86aa6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:27.273254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcb6c29-aa12-4be3-b477-ffee3ef86aa6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:27.274115 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb6c29-aa12-4be3-b477-ffee3ef86aa6 None admin] RBAC: Adding query filter params (role.id=9c81542451e54c0ea22a20c026be6be7, scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d, user.id=b9df4b9d4b0942339277d976457b2124) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:27.274439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb6c29-aa12-4be3-b477-ffee3ef86aa6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:27.275966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcb6c29-aa12-4be3-b477-ffee3ef86aa6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:27.282791 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 45/93] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:27 2022] GET /identity/v3/role_assignments?user.id=b9df4b9d4b0942339277d976457b2124&scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d&role.id=9c81542451e54c0ea22a20c026be6be7 => generated 259 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:27.283690 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb6c29-aa12-4be3-b477-ffee3ef86aa6 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:27.394488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e3814a0-ba91-4a71-9510-7ce1cbfd08e5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:27.407559 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcb6c29-aa12-4be3-b477-ffee3ef86aa6 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:27.432152 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f4d98-9ca8-4c55-b265-4df5c00dda92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:27.433151 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f4d98-9ca8-4c55-b265-4df5c00dda92 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:27.433557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f4d98-9ca8-4c55-b265-4df5c00dda92 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:27.433943 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b6f4d98-9ca8-4c55-b265-4df5c00dda92 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:27.434921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f4d98-9ca8-4c55-b265-4df5c00dda92 None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d, user.id=7effefc613a44b799e8f81fc6a35b678) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:27.435347 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f4d98-9ca8-4c55-b265-4df5c00dda92 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:27.437074 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b6f4d98-9ca8-4c55-b265-4df5c00dda92 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:27.444795 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 49/94] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:27 2022] GET /identity/v3/role_assignments?user.id=7effefc613a44b799e8f81fc6a35b678&scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:27.447084 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b72de7-3b11-4d11-bdd3-f722be71ac03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:27.447779 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b72de7-3b11-4d11-bdd3-f722be71ac03 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:27.448121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b72de7-3b11-4d11-bdd3-f722be71ac03 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:27.448407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b72de7-3b11-4d11-bdd3-f722be71ac03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:27.532625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:27.570289 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b72de7-3b11-4d11-bdd3-f722be71ac03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:27.571863 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3b72de7-3b11-4d11-bdd3-f722be71ac03 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:27.695796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 46/95] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:53:27.701781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f4d98-9ca8-4c55-b265-4df5c00dda92 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:27.728550 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:27.854778 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b6f4d98-9ca8-4c55-b265-4df5c00dda92 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:27.903714 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205ebad-37b5-476f-be6a-6421c09f086d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:27.904673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205ebad-37b5-476f-be6a-6421c09f086d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:27.904974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205ebad-37b5-476f-be6a-6421c09f086d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:27.905266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205ebad-37b5-476f-be6a-6421c09f086d None admin] PATH_INFO: `/v3/projects/e987fcc8b09c46a7a738a5bc966cfe63` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:27.906486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205ebad-37b5-476f-be6a-6421c09f086d None admin] RBAC: Authorizing `identity:get_project(project_id=e987fcc8b09c46a7a738a5bc966cfe63)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:27.908125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5205ebad-37b5-476f-be6a-6421c09f086d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:27.909895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 50/96] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:27 2022] GET /identity/v3/projects/e987fcc8b09c46a7a738a5bc966cfe63 => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:27.916027 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b72de7-3b11-4d11-bdd3-f722be71ac03 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:28.031391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:28.055981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3b72de7-3b11-4d11-bdd3-f722be71ac03 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:28.105962 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac37dd-9245-4457-a20f-5ee7c7bfa33f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:28.106559 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac37dd-9245-4457-a20f-5ee7c7bfa33f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:28.106853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac37dd-9245-4457-a20f-5ee7c7bfa33f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:28.107143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac37dd-9245-4457-a20f-5ee7c7bfa33f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:28.108018 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac37dd-9245-4457-a20f-5ee7c7bfa33f None admin] RBAC: Adding query filter params (role.id=9c81542451e54c0ea22a20c026be6be7, scope.project.id=e987fcc8b09c46a7a738a5bc966cfe63, user.id=b9df4b9d4b0942339277d976457b2124) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:28.108346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac37dd-9245-4457-a20f-5ee7c7bfa33f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:28.109841 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac37dd-9245-4457-a20f-5ee7c7bfa33f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:28.120077 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 47/97] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:27 2022] GET /identity/v3/role_assignments?user.id=b9df4b9d4b0942339277d976457b2124&scope.project.id=e987fcc8b09c46a7a738a5bc966cfe63&role.id=9c81542451e54c0ea22a20c026be6be7 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:29.963943 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f13b3-42dd-4e69-9187-e4613f77fab3 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:29.964511 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f13b3-42dd-4e69-9187-e4613f77fab3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:29.964511 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902f13b3-42dd-4e69-9187-e4613f77fab3 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:29.966099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 51/98] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:29.977161 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6e115-f766-4dbf-b929-8ff37c0fda1e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:29.977595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6e115-f766-4dbf-b929-8ff37c0fda1e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:29.977764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b6e115-f766-4dbf-b929-8ff37c0fda1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:30.005009 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e1b6e115-f766-4dbf-b929-8ff37c0fda1e None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:30.006263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e1b6e115-f766-4dbf-b929-8ff37c0fda1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:30.079579 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 48/99] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:30.114284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f13b3-42dd-4e69-9187-e4613f77fab3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:30.149011 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee612a6-f02c-42cd-98e5-c48c09645f3a None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:30.149339 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee612a6-f02c-42cd-98e5-c48c09645f3a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:30.149590 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee612a6-f02c-42cd-98e5-c48c09645f3a None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:30.151157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 49/100] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:30.166197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad988136-2e27-4ded-8edb-808a318453bd None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:30.166866 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad988136-2e27-4ded-8edb-808a318453bd None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:30.167363 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad988136-2e27-4ded-8edb-808a318453bd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:30.200911 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ad988136-2e27-4ded-8edb-808a318453bd None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:30.202482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ad988136-2e27-4ded-8edb-808a318453bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:30.249175 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:30.282493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902f13b3-42dd-4e69-9187-e4613f77fab3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:30.352545 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77563e16-f91d-44c3-b087-b78900a47c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:30.353825 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77563e16-f91d-44c3-b087-b78900a47c64 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:30.354476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77563e16-f91d-44c3-b087-b78900a47c64 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:30.354978 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77563e16-f91d-44c3-b087-b78900a47c64 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:30.374452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 50/101] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:53:30.380872 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77563e16-f91d-44c3-b087-b78900a47c64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:30.388868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-77563e16-f91d-44c3-b087-b78900a47c64 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:30.388868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:30.388868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:30.388868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:30.388868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:30.388868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:30.388868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:30.388868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:30.388868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:30.388868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:30.388868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:30.388868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:30.388868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:30.388868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:30.388868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:30.388868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:30.389762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:30.389762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:30.389762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:30.389762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:30.389762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:30.389762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:30.389762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:30.389762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:30.389762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:30.389762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:30.389762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:30.389762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:53:30.389762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:30.389762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:30.389762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:30.389762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:30.390573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:30.390573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:30.390573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:30.390573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:30.390573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:30.390573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:30.390573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:30.390573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:30.390573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:30.390573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:30.390573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:30.390573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:53:30.390573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 52/102] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 276 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:30.395921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad988136-2e27-4ded-8edb-808a318453bd None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:30.398812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:30.411455 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77563e16-f91d-44c3-b087-b78900a47c64 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:30.549812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad988136-2e27-4ded-8edb-808a318453bd None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:30.556963 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77563e16-f91d-44c3-b087-b78900a47c64 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:30.594910 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1be72c84-35a3-409d-ad19-07c397b1c834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:30.596010 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be72c84-35a3-409d-ad19-07c397b1c834 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:30.596312 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be72c84-35a3-409d-ad19-07c397b1c834 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:30.596600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1be72c84-35a3-409d-ad19-07c397b1c834 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:30.597369 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be72c84-35a3-409d-ad19-07c397b1c834 None admin] RBAC: Adding query filter params (name=member) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:30.597692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be72c84-35a3-409d-ad19-07c397b1c834 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:30.599103 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1be72c84-35a3-409d-ad19-07c397b1c834 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:30.600786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b7b749-ea3c-4a57-af16-deebbc8acd79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:30.601834 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7b749-ea3c-4a57-af16-deebbc8acd79 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:30.602337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7b749-ea3c-4a57-af16-deebbc8acd79 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:30.602823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b7b749-ea3c-4a57-af16-deebbc8acd79 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:30.604251 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7b749-ea3c-4a57-af16-deebbc8acd79 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:30.605931 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b7b749-ea3c-4a57-af16-deebbc8acd79 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:30.607835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 51/103] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:30.608209 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 53/104] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:53:30 2022] GET /identity/v3/domains/default => generated 199 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:30.609626 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b2f10-b1c4-4a01-a396-ac78cc5871bc None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:30.609820 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da58a47-7b5a-495d-91d4-3f2ab87904e5 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:30.610218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da58a47-7b5a-495d-91d4-3f2ab87904e5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:30.610376 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b2f10-b1c4-4a01-a396-ac78cc5871bc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:30.610736 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da58a47-7b5a-495d-91d4-3f2ab87904e5 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:30.611045 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661b2f10-b1c4-4a01-a396-ac78cc5871bc None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:30.612638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 52/105] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:30.613476 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b2f10-b1c4-4a01-a396-ac78cc5871bc None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:30.616283 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 54/106] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:30 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:30.619867 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da58a47-7b5a-495d-91d4-3f2ab87904e5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:30.632206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:30.687794 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:30.724084 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3da58a47-7b5a-495d-91d4-3f2ab87904e5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:30.729644 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661b2f10-b1c4-4a01-a396-ac78cc5871bc None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:30.758620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd69b806-9384-4b15-b878-11661a9db4a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:30.759900 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69b806-9384-4b15-b878-11661a9db4a4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:30.760397 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69b806-9384-4b15-b878-11661a9db4a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:30.760873 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd69b806-9384-4b15-b878-11661a9db4a4 None admin] PATH_INFO: `/v3/users/b9df4b9d4b0942339277d976457b2124` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:30.762469 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69b806-9384-4b15-b878-11661a9db4a4 None admin] RBAC: Authorizing `identity:get_user(user_id=b9df4b9d4b0942339277d976457b2124)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:30.764597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd69b806-9384-4b15-b878-11661a9db4a4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:30.765181 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f49ba22-8bd1-47c8-bbda-1b6ec12885a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:30.765932 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49ba22-8bd1-47c8-bbda-1b6ec12885a1 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:30.766315 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49ba22-8bd1-47c8-bbda-1b6ec12885a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:30.766722 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f49ba22-8bd1-47c8-bbda-1b6ec12885a1 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:30.767679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49ba22-8bd1-47c8-bbda-1b6ec12885a1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:30.767839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 55/107] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:30 2022] GET /identity/v3/users/b9df4b9d4b0942339277d976457b2124 => generated 275 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:30.769293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034a19e-8d72-4273-b5a8-1006964419a8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:30.769623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f49ba22-8bd1-47c8-bbda-1b6ec12885a1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:30.769748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034a19e-8d72-4273-b5a8-1006964419a8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:30.770123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7034a19e-8d72-4273-b5a8-1006964419a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:30.787766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7034a19e-8d72-4273-b5a8-1006964419a8 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:30.789188 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7034a19e-8d72-4273-b5a8-1006964419a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:30.808665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 53/108] 10.222.0.85 () {60 vars in 1141 bytes} [Wed May 25 15:53:30 2022] POST /identity/v3/projects => generated 302 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:53:30.810677 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4ce76-5715-4047-a5aa-1b85edb2474b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:30.811535 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4ce76-5715-4047-a5aa-1b85edb2474b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:30.812067 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27f4ce76-5715-4047-a5aa-1b85edb2474b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:30.837053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-27f4ce76-5715-4047-a5aa-1b85edb2474b None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:30.838597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-27f4ce76-5715-4047-a5aa-1b85edb2474b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:30.878334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 56/109] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:30.880116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034a19e-8d72-4273-b5a8-1006964419a8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:30.884127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:30.993033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 54/110] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:30.994024 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f4ce76-5715-4047-a5aa-1b85edb2474b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:31.000356 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7034a19e-8d72-4273-b5a8-1006964419a8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:31.056791 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c98132-771b-4129-91a4-e46c94ec4b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:31.057689 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c98132-771b-4129-91a4-e46c94ec4b37 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:31.057982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c98132-771b-4129-91a4-e46c94ec4b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:31.058270 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c98132-771b-4129-91a4-e46c94ec4b37 None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:31.059689 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c98132-771b-4129-91a4-e46c94ec4b37 None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:31.061185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c98132-771b-4129-91a4-e46c94ec4b37 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:31.063311 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 57/111] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:30 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:31.068317 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:31.069880 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60695424-a924-4ef8-9b2c-3f52fc2c5378 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:31.070231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60695424-a924-4ef8-9b2c-3f52fc2c5378 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:31.070514 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60695424-a924-4ef8-9b2c-3f52fc2c5378 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:31.076221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 58/112] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:31 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:31.077245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60695424-a924-4ef8-9b2c-3f52fc2c5378 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:31.139287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27f4ce76-5715-4047-a5aa-1b85edb2474b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:31.167205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:31.182056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3e89b-97ad-4431-93f0-8b41905b18ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:31.183212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3e89b-97ad-4431-93f0-8b41905b18ed None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:31.183510 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3e89b-97ad-4431-93f0-8b41905b18ed None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:31.183797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fe3e89b-97ad-4431-93f0-8b41905b18ed None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:31.198938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60695424-a924-4ef8-9b2c-3f52fc2c5378 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:31.203393 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fe3e89b-97ad-4431-93f0-8b41905b18ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:31.213797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-9fe3e89b-97ad-4431-93f0-8b41905b18ed None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:53:31.213797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:31.213797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:31.213797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:31.213797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:31.213797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:31.213797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:31.213797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:31.213797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:31.213797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:31.213797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:31.213797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:31.213797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:31.213797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:31.213797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:31.213797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:31.214668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:31.214668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:31.214668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:31.214668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:31.214668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:31.214668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:31.214668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:31.214668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:31.214668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:31.214668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:31.214668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:31.214668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:31.214668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:31.214668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:31.214668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:31.214668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:31.215433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:31.215433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:31.215433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:31.215433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:31.215433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:31.215433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:31.215433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:31.215433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:31.215433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:31.215433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:31.215433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:53:31.215433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:31.216047 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 55/113] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:53:30 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:31.221121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3e89b-97ad-4431-93f0-8b41905b18ed None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:31.242660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-054ab594-4a02-4c7b-b2db-f5fb01db78b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:31.243788 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ab594-4a02-4c7b-b2db-f5fb01db78b6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:31.244252 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ab594-4a02-4c7b-b2db-f5fb01db78b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:31.244606 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-054ab594-4a02-4c7b-b2db-f5fb01db78b6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:31.258147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-054ab594-4a02-4c7b-b2db-f5fb01db78b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:31.272541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-054ab594-4a02-4c7b-b2db-f5fb01db78b6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:53:31.272541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:31.272541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:31.272541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:31.272541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:31.272541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:31.272541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:31.272541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:31.272541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:31.272541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:31.272541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:31.272541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:31.272541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:31.272541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:31.272541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:31.272541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:31.274552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:31.274552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:31.274552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:31.274552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:31.274552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:31.274552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:31.274552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:31.274552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:31.274552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:31.274552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:31.274552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:31.274552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:53:31.274552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:31.274552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:31.274552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:31.274552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:31.276413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:31.276413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:31.276413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:31.276413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:31.276413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:31.276413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:31.276413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:31.276413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:31.276413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:31.276413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:31.276413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:53:31.276413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:53:31.276413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 59/114] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:31 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:31.276413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603aa46d-9702-4d8c-9c5c-7ab96250ce52 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:31.278115 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603aa46d-9702-4d8c-9c5c-7ab96250ce52 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:31.278115 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603aa46d-9702-4d8c-9c5c-7ab96250ce52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:31.303845 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-603aa46d-9702-4d8c-9c5c-7ab96250ce52 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:31.303845 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-603aa46d-9702-4d8c-9c5c-7ab96250ce52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:31.334433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fe3e89b-97ad-4431-93f0-8b41905b18ed None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:31.342727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:31.375317 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe0031-ff38-4320-b2a2-49d5e4c1d30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:31.376776 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe0031-ff38-4320-b2a2-49d5e4c1d30d None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:31.377288 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe0031-ff38-4320-b2a2-49d5e4c1d30d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:31.377728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fe0031-ff38-4320-b2a2-49d5e4c1d30d None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d/users/b9df4b9d4b0942339277d976457b2124/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:31.380484 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe0031-ff38-4320-b2a2-49d5e4c1d30d None admin] RBAC: Authorizing `identity:create_grant(project_id=89a647c5dfc34a43ad12c49f8f57d99d, user_id=b9df4b9d4b0942339277d976457b2124, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:31.382241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fe0031-ff38-4320-b2a2-49d5e4c1d30d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:31.384249 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 60/115] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:31.385501 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603aa46d-9702-4d8c-9c5c-7ab96250ce52 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:31.414615 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 56/116] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:53:31 2022] PUT /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d/users/b9df4b9d4b0942339277d976457b2124/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:53:31.415493 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe0031-ff38-4320-b2a2-49d5e4c1d30d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:31.466184 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:31.511988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603aa46d-9702-4d8c-9c5c-7ab96250ce52 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:31.552219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93eba1-a338-40bd-8729-501cd2c75a9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:31.552973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93eba1-a338-40bd-8729-501cd2c75a9d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:31.553268 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93eba1-a338-40bd-8729-501cd2c75a9d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:31.553560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c93eba1-a338-40bd-8729-501cd2c75a9d None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:31.554527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93eba1-a338-40bd-8729-501cd2c75a9d None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:31.554855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93eba1-a338-40bd-8729-501cd2c75a9d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:31.556421 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c93eba1-a338-40bd-8729-501cd2c75a9d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:31.566596 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 61/117] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:53:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:31.567650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93eba1-a338-40bd-8729-501cd2c75a9d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:31.624299 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fe0031-ff38-4320-b2a2-49d5e4c1d30d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:31.676951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17c7cc-a4c8-4216-9c66-8177c6dafa79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:31.678213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17c7cc-a4c8-4216-9c66-8177c6dafa79 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:31.678666 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17c7cc-a4c8-4216-9c66-8177c6dafa79 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:31.679545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17c7cc-a4c8-4216-9c66-8177c6dafa79 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:31.680547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17c7cc-a4c8-4216-9c66-8177c6dafa79 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:31.681038 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17c7cc-a4c8-4216-9c66-8177c6dafa79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:31.683300 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17c7cc-a4c8-4216-9c66-8177c6dafa79 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:31.693774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 57/118] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:31 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:31.694649 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17c7cc-a4c8-4216-9c66-8177c6dafa79 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:31.698992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c93eba1-a338-40bd-8729-501cd2c75a9d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:31.725069 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:31.728282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:31.738673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8e0a456-fce1-4445-92bd-9b18d6878c00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:31.739536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0a456-fce1-4445-92bd-9b18d6878c00 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:31.739860 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0a456-fce1-4445-92bd-9b18d6878c00 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:31.740153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e0a456-fce1-4445-92bd-9b18d6878c00 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:31.750947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8e0a456-fce1-4445-92bd-9b18d6878c00 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:31.758808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b8e0a456-fce1-4445-92bd-9b18d6878c00 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:31.758808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:31.758808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:31.758808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:31.758808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:31.758808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:31.758808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:31.758808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:31.758808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:31.758808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:31.758808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:31.758808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:31.758808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:31.758808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:31.758808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:31.758808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:31.759681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:31.759681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:31.759681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:31.759681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:31.759681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:31.759681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:31.759681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:31.759681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:31.759681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:31.759681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:31.759681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:31.759681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:53:31.759681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:31.759681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:31.759681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:31.759681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:31.760476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:31.760476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:31.760476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:31.760476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:31.760476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:31.760476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:31.760476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:31.760476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:31.760476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:31.760476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:31.760476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:31.760476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:53:31.760476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 62/119] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:53:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:31.764996 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dbfd5-97a3-45fe-80e6-a4ed63c4ea3d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:31.764996 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dbfd5-97a3-45fe-80e6-a4ed63c4ea3d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:31.764996 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd7dbfd5-97a3-45fe-80e6-a4ed63c4ea3d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:31.764996 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 63/120] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:31 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:31.768110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7dbfd5-97a3-45fe-80e6-a4ed63c4ea3d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:31.813027 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17c7cc-a4c8-4216-9c66-8177c6dafa79 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:31.848631 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ae410-d408-41e9-90c3-400be5eaa167 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:31.849569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ae410-d408-41e9-90c3-400be5eaa167 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:31.849864 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ae410-d408-41e9-90c3-400be5eaa167 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:31.850152 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1ae410-d408-41e9-90c3-400be5eaa167 None admin] PATH_INFO: `/v3/users/b9df4b9d4b0942339277d976457b2124` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:31.851373 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ae410-d408-41e9-90c3-400be5eaa167 None admin] RBAC: Authorizing `identity:get_user(user_id=b9df4b9d4b0942339277d976457b2124)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:31.852900 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba1ae410-d408-41e9-90c3-400be5eaa167 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:31.854930 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 58/121] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:31 2022] GET /identity/v3/users/b9df4b9d4b0942339277d976457b2124 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:31.855780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ae410-d408-41e9-90c3-400be5eaa167 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:31.890106 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd7dbfd5-97a3-45fe-80e6-a4ed63c4ea3d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:31.929008 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db731865-36a6-4605-a886-39c35f7a506e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:31.930154 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db731865-36a6-4605-a886-39c35f7a506e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:31.930449 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db731865-36a6-4605-a886-39c35f7a506e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:31.930733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db731865-36a6-4605-a886-39c35f7a506e None admin] PATH_INFO: `/v3/users/50d9b6abb4e3401e883c58efc08b6eed` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:31.948376 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db731865-36a6-4605-a886-39c35f7a506e None admin] RBAC: Authorizing `identity:get_user(user_id=50d9b6abb4e3401e883c58efc08b6eed)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:31.950357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db731865-36a6-4605-a886-39c35f7a506e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:31.952818 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 64/122] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:31 2022] GET /identity/v3/users/50d9b6abb4e3401e883c58efc08b6eed => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:31.954252 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e0eec-6866-4f37-a8b8-8118bcc4e36d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:31.954613 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e0eec-6866-4f37-a8b8-8118bcc4e36d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:31.954910 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f98e0eec-6866-4f37-a8b8-8118bcc4e36d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:31.965025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:31.971902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1ae410-d408-41e9-90c3-400be5eaa167 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:31.977702 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f98e0eec-6866-4f37-a8b8-8118bcc4e36d None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:31.978896 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f98e0eec-6866-4f37-a8b8-8118bcc4e36d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:32.010088 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e897c-2e3d-4da4-bb88-754a2f231bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:32.011604 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e897c-2e3d-4da4-bb88-754a2f231bc6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:32.012158 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e897c-2e3d-4da4-bb88-754a2f231bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:32.012618 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18e897c-2e3d-4da4-bb88-754a2f231bc6 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:32.013522 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e897c-2e3d-4da4-bb88-754a2f231bc6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:32.014091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e897c-2e3d-4da4-bb88-754a2f231bc6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:32.015789 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18e897c-2e3d-4da4-bb88-754a2f231bc6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:32.023010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:32.034867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 59/123] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:31 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:32.036085 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e897c-2e3d-4da4-bb88-754a2f231bc6 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:32.064052 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 65/124] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:32.070834 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e0eec-6866-4f37-a8b8-8118bcc4e36d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:32.167524 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18e897c-2e3d-4da4-bb88-754a2f231bc6 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:32.196919 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f98e0eec-6866-4f37-a8b8-8118bcc4e36d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:32.207842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ff89c-0404-47bf-a90b-3b11ec248973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:32.209287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ff89c-0404-47bf-a90b-3b11ec248973 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:32.209677 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ff89c-0404-47bf-a90b-3b11ec248973 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:32.210035 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a2ff89c-0404-47bf-a90b-3b11ec248973 None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:32.212081 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ff89c-0404-47bf-a90b-3b11ec248973 None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:32.214134 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a2ff89c-0404-47bf-a90b-3b11ec248973 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:32.217023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 60/125] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:32 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:32.218187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ff89c-0404-47bf-a90b-3b11ec248973 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:32.240667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d826d1e-885c-4908-bd11-262fe601d80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:32.241646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d826d1e-885c-4908-bd11-262fe601d80f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:32.242145 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d826d1e-885c-4908-bd11-262fe601d80f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:32.242602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d826d1e-885c-4908-bd11-262fe601d80f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:32.252064 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:32.288401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:32.335398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d826d1e-885c-4908-bd11-262fe601d80f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:32.337289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d826d1e-885c-4908-bd11-262fe601d80f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:32.351690 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a2ff89c-0404-47bf-a90b-3b11ec248973 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:32.399171 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f30a07-63a2-46b2-a159-d123344217a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:32.400071 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f30a07-63a2-46b2-a159-d123344217a3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:32.400365 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f30a07-63a2-46b2-a159-d123344217a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:32.400651 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f30a07-63a2-46b2-a159-d123344217a3 None admin] PATH_INFO: `/v3/users/7effefc613a44b799e8f81fc6a35b678` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:32.402274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f30a07-63a2-46b2-a159-d123344217a3 None admin] RBAC: Authorizing `identity:get_user(user_id=7effefc613a44b799e8f81fc6a35b678)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:32.404056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f30a07-63a2-46b2-a159-d123344217a3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:32.405878 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 61/126] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:32 2022] GET /identity/v3/users/7effefc613a44b799e8f81fc6a35b678 => generated 247 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:32.406652 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f30a07-63a2-46b2-a159-d123344217a3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:32.478582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 66/127] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:53:32.480037 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d826d1e-885c-4908-bd11-262fe601d80f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:32.520321 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:32.524797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f30a07-63a2-46b2-a159-d123344217a3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:32.566567 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdd0a6-661b-4fce-a71c-8ace5e28fee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:32.567661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdd0a6-661b-4fce-a71c-8ace5e28fee8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:32.567992 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdd0a6-661b-4fce-a71c-8ace5e28fee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:32.568282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cdd0a6-661b-4fce-a71c-8ace5e28fee8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:32.570523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:32.582363 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cdd0a6-661b-4fce-a71c-8ace5e28fee8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:32.591290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-a4cdd0a6-661b-4fce-a71c-8ace5e28fee8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:32.591290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:32.591290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:32.591290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:32.591290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:32.591290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:32.591290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:32.591290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:32.591290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:32.591290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:32.591290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:32.591290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:32.591290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:32.591290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:32.591290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:32.591290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:32.592439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:32.592439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:32.592439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:32.592439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:32.592439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:32.592439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:32.592439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:32.592439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:32.592439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:32.592439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:32.592439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:32.592439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:32.592439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:32.592439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:32.592439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:32.592439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:32.593414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:32.593414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:32.593414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:32.593414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:32.593414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:32.593414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:32.593414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:32.593414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:32.593414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:32.593414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:32.593414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:32.593414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:32.595945 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 62/128] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:32.595945 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdd0a6-661b-4fce-a71c-8ace5e28fee8 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:32.622366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d826d1e-885c-4908-bd11-262fe601d80f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:32.667762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3f4d1-bd72-4641-9601-73d22e64064c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:32.669015 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3f4d1-bd72-4641-9601-73d22e64064c None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:32.669310 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3f4d1-bd72-4641-9601-73d22e64064c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:32.669597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b3f4d1-bd72-4641-9601-73d22e64064c None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d/users/b9df4b9d4b0942339277d976457b2124/roles/1cc9b5d798c648c986537d006320581f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:32.672698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b3f4d1-bd72-4641-9601-73d22e64064c None admin] RBAC: Authorizing `identity:create_grant(project_id=89a647c5dfc34a43ad12c49f8f57d99d, user_id=b9df4b9d4b0942339277d976457b2124, role_id=1cc9b5d798c648c986537d006320581f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:32.674648 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b3f4d1-bd72-4641-9601-73d22e64064c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:32.694118 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 67/129] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:53:32 2022] PUT /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d/users/b9df4b9d4b0942339277d976457b2124/roles/1cc9b5d798c648c986537d006320581f => generated 0 bytes in 215 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:53:32.695659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3f4d1-bd72-4641-9601-73d22e64064c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:32.729627 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cdd0a6-661b-4fce-a71c-8ace5e28fee8 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:32.771282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc281f68-f15f-4568-a9f8-63b6e5c2dc3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:32.772159 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc281f68-f15f-4568-a9f8-63b6e5c2dc3b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:32.772452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc281f68-f15f-4568-a9f8-63b6e5c2dc3b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:32.772741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc281f68-f15f-4568-a9f8-63b6e5c2dc3b None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:32.774172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc281f68-f15f-4568-a9f8-63b6e5c2dc3b None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:32.776027 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc281f68-f15f-4568-a9f8-63b6e5c2dc3b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:32.778367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 63/130] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:32 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:32.779246 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc281f68-f15f-4568-a9f8-63b6e5c2dc3b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:32.810133 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:32.823428 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b3f4d1-bd72-4641-9601-73d22e64064c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:32.858415 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:32.866083 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad4586-0c89-4556-be73-867112ab9a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:32.867347 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad4586-0c89-4556-be73-867112ab9a8c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:32.868034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad4586-0c89-4556-be73-867112ab9a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:32.868614 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ad4586-0c89-4556-be73-867112ab9a8c None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:32.870610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad4586-0c89-4556-be73-867112ab9a8c None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:32.874129 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ad4586-0c89-4556-be73-867112ab9a8c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:32.876601 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 68/131] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:32 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:32.877686 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad4586-0c89-4556-be73-867112ab9a8c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:32.908156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc281f68-f15f-4568-a9f8-63b6e5c2dc3b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:32.952503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76cded-fc23-425e-a6df-85e9d040328b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:32.953241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76cded-fc23-425e-a6df-85e9d040328b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:32.953536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76cded-fc23-425e-a6df-85e9d040328b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:32.953816 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca76cded-fc23-425e-a6df-85e9d040328b None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:32.954784 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca76cded-fc23-425e-a6df-85e9d040328b None admin] RBAC: Adding query filter params (name=member) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:32.956697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca76cded-fc23-425e-a6df-85e9d040328b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:32.957448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca76cded-fc23-425e-a6df-85e9d040328b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:32.968305 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 64/132] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:32.969348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76cded-fc23-425e-a6df-85e9d040328b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:33.012152 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ad4586-0c89-4556-be73-867112ab9a8c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:33.060909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cb8f3-b631-4d5c-b292-db23b1fd4607 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:33.062649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cb8f3-b631-4d5c-b292-db23b1fd4607 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:33.063150 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cb8f3-b631-4d5c-b292-db23b1fd4607 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:33.063667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cb8f3-b631-4d5c-b292-db23b1fd4607 None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d/users/7effefc613a44b799e8f81fc6a35b678/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:33.065955 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cb8f3-b631-4d5c-b292-db23b1fd4607 None admin] RBAC: Authorizing `identity:create_grant(project_id=89a647c5dfc34a43ad12c49f8f57d99d, user_id=7effefc613a44b799e8f81fc6a35b678, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:33.067674 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cb8f3-b631-4d5c-b292-db23b1fd4607 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:33.082280 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 69/133] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:53:32 2022] PUT /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d/users/7effefc613a44b799e8f81fc6a35b678/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:53:33.083464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cb8f3-b631-4d5c-b292-db23b1fd4607 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:33.097500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:33.116811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca76cded-fc23-425e-a6df-85e9d040328b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:33.159416 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897dd119-20f0-4fd3-a812-92ad4c29d3e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:33.160747 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897dd119-20f0-4fd3-a812-92ad4c29d3e8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:33.160747 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897dd119-20f0-4fd3-a812-92ad4c29d3e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:33.160747 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897dd119-20f0-4fd3-a812-92ad4c29d3e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:33.161947 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897dd119-20f0-4fd3-a812-92ad4c29d3e8 None admin] RBAC: Adding query filter params (role.id=b7283d3771c94f458ebfcaef5784afa0, scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d, user.id=50d9b6abb4e3401e883c58efc08b6eed) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:33.162276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897dd119-20f0-4fd3-a812-92ad4c29d3e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:33.164094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897dd119-20f0-4fd3-a812-92ad4c29d3e8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:33.169066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:33.170798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 65/134] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:32 2022] GET /identity/v3/role_assignments?user.id=50d9b6abb4e3401e883c58efc08b6eed&scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d&role.id=b7283d3771c94f458ebfcaef5784afa0 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:33.233278 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cb8f3-b631-4d5c-b292-db23b1fd4607 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:33.281397 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8db53b-f40c-488d-9c82-7da18aefaf80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:33.282794 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8db53b-f40c-488d-9c82-7da18aefaf80 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:33.283311 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8db53b-f40c-488d-9c82-7da18aefaf80 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:33.283794 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8db53b-f40c-488d-9c82-7da18aefaf80 None admin] PATH_INFO: `/v3/users/b9df4b9d4b0942339277d976457b2124` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:33.285398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8db53b-f40c-488d-9c82-7da18aefaf80 None admin] RBAC: Authorizing `identity:get_user(user_id=b9df4b9d4b0942339277d976457b2124)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:33.287121 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8db53b-f40c-488d-9c82-7da18aefaf80 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:33.289126 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 70/135] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:33 2022] GET /identity/v3/users/b9df4b9d4b0942339277d976457b2124 => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:33.298238 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897dd119-20f0-4fd3-a812-92ad4c29d3e8 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:33.460142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897dd119-20f0-4fd3-a812-92ad4c29d3e8 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:33.500435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57eba056-37e5-48cd-bfa8-f286e34d21e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:33.501373 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eba056-37e5-48cd-bfa8-f286e34d21e8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:33.501662 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eba056-37e5-48cd-bfa8-f286e34d21e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:33.501941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57eba056-37e5-48cd-bfa8-f286e34d21e8 None admin] PATH_INFO: `/v3/projects/e987fcc8b09c46a7a738a5bc966cfe63` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:33.503189 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eba056-37e5-48cd-bfa8-f286e34d21e8 None admin] RBAC: Authorizing `identity:get_project(project_id=e987fcc8b09c46a7a738a5bc966cfe63)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:33.504687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57eba056-37e5-48cd-bfa8-f286e34d21e8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:33.506714 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 66/136] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:33 2022] GET /identity/v3/projects/e987fcc8b09c46a7a738a5bc966cfe63 => generated 312 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:33.512986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8db53b-f40c-488d-9c82-7da18aefaf80 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:33.597623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a458509-250f-4612-9397-abf0aba2cf6f None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:33.597915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a458509-250f-4612-9397-abf0aba2cf6f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:33.598123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a458509-250f-4612-9397-abf0aba2cf6f None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:33.599710 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 67/137] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:53:33.607090 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325def4-6c28-47c9-be5e-adeb42c94892 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:33.607888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325def4-6c28-47c9-be5e-adeb42c94892 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:33.607888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f325def4-6c28-47c9-be5e-adeb42c94892 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:33.628734 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f325def4-6c28-47c9-be5e-adeb42c94892 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:33.629928 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f325def4-6c28-47c9-be5e-adeb42c94892 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:33.643436 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:33.644014 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8db53b-f40c-488d-9c82-7da18aefaf80 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:33.693161 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:33.712984 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 68/138] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:53:33.716918 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a8c1a-b43c-462c-bbe4-d0d0c1667137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:33.718114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a8c1a-b43c-462c-bbe4-d0d0c1667137 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:33.718616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a8c1a-b43c-462c-bbe4-d0d0c1667137 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:33.719095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270a8c1a-b43c-462c-bbe4-d0d0c1667137 None admin] PATH_INFO: `/v3/projects/e987fcc8b09c46a7a738a5bc966cfe63/users/b9df4b9d4b0942339277d976457b2124/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:33.721373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a8c1a-b43c-462c-bbe4-d0d0c1667137 None admin] RBAC: Authorizing `identity:create_grant(project_id=e987fcc8b09c46a7a738a5bc966cfe63, user_id=b9df4b9d4b0942339277d976457b2124, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:33.723088 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270a8c1a-b43c-462c-bbe4-d0d0c1667137 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:33.738706 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325def4-6c28-47c9-be5e-adeb42c94892 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:33.755368 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 71/139] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:53:33 2022] PUT /identity/v3/projects/e987fcc8b09c46a7a738a5bc966cfe63/users/b9df4b9d4b0942339277d976457b2124/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:53:33.847137 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f325def4-6c28-47c9-be5e-adeb42c94892 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:33.883036 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee02a50-062c-4add-8e9c-c5ccb9af7597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:33.883896 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee02a50-062c-4add-8e9c-c5ccb9af7597 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:33.884190 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee02a50-062c-4add-8e9c-c5ccb9af7597 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:33.884474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee02a50-062c-4add-8e9c-c5ccb9af7597 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:33.885843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee02a50-062c-4add-8e9c-c5ccb9af7597 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:33.887397 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee02a50-062c-4add-8e9c-c5ccb9af7597 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:33.893770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 69/140] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:53:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:33.895460 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a8c1a-b43c-462c-bbe4-d0d0c1667137 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:34.021639 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270a8c1a-b43c-462c-bbe4-d0d0c1667137 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:34.070400 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2969bfeb-adff-4f7d-9694-d152f58942c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:34.071157 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969bfeb-adff-4f7d-9694-d152f58942c8 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:34.071373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969bfeb-adff-4f7d-9694-d152f58942c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:34.071652 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2969bfeb-adff-4f7d-9694-d152f58942c8 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:34.072426 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969bfeb-adff-4f7d-9694-d152f58942c8 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:34.073995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2969bfeb-adff-4f7d-9694-d152f58942c8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:34.109867 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 72/141] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:53:33 2022] POST /identity/v3/users => generated 283 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:53:34.458525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637f80b-f7fb-404d-aff8-009ea6abd5df None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:34.459034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637f80b-f7fb-404d-aff8-009ea6abd5df None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:34.459301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6637f80b-f7fb-404d-aff8-009ea6abd5df None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:34.460713 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 70/142] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:34.466794 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfb37a-9824-499c-b285-b006baf32d96 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:34.467411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfb37a-9824-499c-b285-b006baf32d96 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:34.467918 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ccfb37a-9824-499c-b285-b006baf32d96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:34.491679 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0ccfb37a-9824-499c-b285-b006baf32d96 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:34.493229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0ccfb37a-9824-499c-b285-b006baf32d96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:34.537185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:34.566525 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 73/143] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:34.589639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637f80b-f7fb-404d-aff8-009ea6abd5df None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:34.606200 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:34.695055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6637f80b-f7fb-404d-aff8-009ea6abd5df None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:34.730438 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841bdbb-fad3-472b-a823-0634b69378ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:34.731610 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841bdbb-fad3-472b-a823-0634b69378ea None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:34.732128 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841bdbb-fad3-472b-a823-0634b69378ea None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:34.732596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8841bdbb-fad3-472b-a823-0634b69378ea None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:34.743491 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8841bdbb-fad3-472b-a823-0634b69378ea None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:34.751859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-8841bdbb-fad3-472b-a823-0634b69378ea None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:34.751859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:34.751859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:34.751859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:34.751859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:34.751859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:34.751859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:34.751859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:34.751859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:34.751859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:34.751859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:34.751859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:34.751859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:34.751859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:34.751859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:34.751859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:34.752823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:34.752823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:34.752823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:34.752823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:34.752823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:34.752823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:34.752823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:34.752823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:34.752823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:34.752823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:34.752823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:34.752823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:34.752823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:34.752823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:34.752823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:34.752823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:34.753578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:34.753578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:34.753578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:34.753578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:34.753578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:34.753578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:34.753578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:34.753578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:34.753578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:34.753578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:34.753578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:34.753578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:34.756023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 71/144] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:34 2022] GET /identity/v3/roles/member => generated 84 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:34.762102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfb37a-9824-499c-b285-b006baf32d96 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:34.874365 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ccfb37a-9824-499c-b285-b006baf32d96 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:34.913969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d34c90-a824-468b-a564-bd5acd55d8f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:34.914982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d34c90-a824-468b-a564-bd5acd55d8f8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:34.915488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d34c90-a824-468b-a564-bd5acd55d8f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:34.915991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d34c90-a824-468b-a564-bd5acd55d8f8 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:34.916940 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d34c90-a824-468b-a564-bd5acd55d8f8 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:34.917453 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d34c90-a824-468b-a564-bd5acd55d8f8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:34.919114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d34c90-a824-468b-a564-bd5acd55d8f8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:34.928633 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 74/145] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:34 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:34.934551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841bdbb-fad3-472b-a823-0634b69378ea None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:35.014302 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:35.040561 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8841bdbb-fad3-472b-a823-0634b69378ea None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:35.075078 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558547cb-2ac5-4c32-bdc9-24a1b2fd0432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:35.076469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558547cb-2ac5-4c32-bdc9-24a1b2fd0432 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:35.076972 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558547cb-2ac5-4c32-bdc9-24a1b2fd0432 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:35.077458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-558547cb-2ac5-4c32-bdc9-24a1b2fd0432 None admin] PATH_INFO: `/v3/users/b9df4b9d4b0942339277d976457b2124` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:35.078852 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558547cb-2ac5-4c32-bdc9-24a1b2fd0432 None admin] RBAC: Authorizing `identity:get_user(user_id=b9df4b9d4b0942339277d976457b2124)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:35.080707 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-558547cb-2ac5-4c32-bdc9-24a1b2fd0432 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:35.082579 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 72/146] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:34 2022] GET /identity/v3/users/b9df4b9d4b0942339277d976457b2124 => generated 275 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:35.088625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d34c90-a824-468b-a564-bd5acd55d8f8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:35.127516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:35.198315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d34c90-a824-468b-a564-bd5acd55d8f8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:35.243178 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04d37b-a0c7-4fa3-8b4b-9900925aa60a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:35.244550 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04d37b-a0c7-4fa3-8b4b-9900925aa60a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:35.245093 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04d37b-a0c7-4fa3-8b4b-9900925aa60a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:35.245559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb04d37b-a0c7-4fa3-8b4b-9900925aa60a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:35.326697 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04d37b-a0c7-4fa3-8b4b-9900925aa60a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:35.328216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb04d37b-a0c7-4fa3-8b4b-9900925aa60a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:35.366614 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:35.443005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 75/147] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:53:35.448695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558547cb-2ac5-4c32-bdc9-24a1b2fd0432 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:35.573495 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-558547cb-2ac5-4c32-bdc9-24a1b2fd0432 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:35.611737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4a94ae-4989-4a0c-b759-b27487ef26e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:35.612698 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a94ae-4989-4a0c-b759-b27487ef26e1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:35.612983 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a94ae-4989-4a0c-b759-b27487ef26e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:35.613286 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4a94ae-4989-4a0c-b759-b27487ef26e1 None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:35.615066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a94ae-4989-4a0c-b759-b27487ef26e1 None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:35.616660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4a94ae-4989-4a0c-b759-b27487ef26e1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:35.618745 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 73/148] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:35 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:35.624559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04d37b-a0c7-4fa3-8b4b-9900925aa60a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:35.733894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb04d37b-a0c7-4fa3-8b4b-9900925aa60a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:35.771420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834a7572-c0d8-4ad2-b998-1b9dfe81fb34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:35.772128 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a7572-c0d8-4ad2-b998-1b9dfe81fb34 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:35.772612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a7572-c0d8-4ad2-b998-1b9dfe81fb34 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:35.772895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834a7572-c0d8-4ad2-b998-1b9dfe81fb34 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:35.773749 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a7572-c0d8-4ad2-b998-1b9dfe81fb34 None admin] RBAC: Adding query filter params (role.id=9c81542451e54c0ea22a20c026be6be7, scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d, user.id=b9df4b9d4b0942339277d976457b2124) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:35.774060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a7572-c0d8-4ad2-b998-1b9dfe81fb34 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:35.775544 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-834a7572-c0d8-4ad2-b998-1b9dfe81fb34 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:35.779333 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:35.783311 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 76/149] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:35 2022] GET /identity/v3/role_assignments?user.id=b9df4b9d4b0942339277d976457b2124&scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d&role.id=9c81542451e54c0ea22a20c026be6be7 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:36.083022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81ad4e-e17a-41a0-81e0-67c25a949c3e None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:36.083385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81ad4e-e17a-41a0-81e0-67c25a949c3e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:36.083572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b81ad4e-e17a-41a0-81e0-67c25a949c3e None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:36.085220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 74/150] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:36.091530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0812f-8ebc-48d1-ae86-0132a10f78b8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:36.092077 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0812f-8ebc-48d1-ae86-0132a10f78b8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:36.092150 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21b0812f-8ebc-48d1-ae86-0132a10f78b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:36.121107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-21b0812f-8ebc-48d1-ae86-0132a10f78b8 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:36.122300 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-21b0812f-8ebc-48d1-ae86-0132a10f78b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:36.215858 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 77/151] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:53:36.251546 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81ad4e-e17a-41a0-81e0-67c25a949c3e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:36.264836 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:36.373217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b81ad4e-e17a-41a0-81e0-67c25a949c3e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:36.410807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8b816-4669-4164-a8c0-53294b3588d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:36.411691 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8b816-4669-4164-a8c0-53294b3588d7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:36.412007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8b816-4669-4164-a8c0-53294b3588d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:36.412285 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8b816-4669-4164-a8c0-53294b3588d7 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:36.423103 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8b816-4669-4164-a8c0-53294b3588d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:36.431518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-47d8b816-4669-4164-a8c0-53294b3588d7 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:53:36.431518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:36.431518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:36.431518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:36.431518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:36.431518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:36.431518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:36.431518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:36.431518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:36.431518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:36.431518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:36.431518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:36.431518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:36.431518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:36.431518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:36.431518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:36.432407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:36.432407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:36.432407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:36.432407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:36.432407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:36.432407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:36.432407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:36.432407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:36.432407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:36.432407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:36.432407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:36.432407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:36.432407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:36.432407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:36.432407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:36.432407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:36.433185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:36.433185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:36.433185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:36.433185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:36.433185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:36.433185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:36.433185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:36.433185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:36.433185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:36.433185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:36.433185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:53:36.433185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:36.433185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 75/152] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:53:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:36.439809 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0812f-8ebc-48d1-ae86-0132a10f78b8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:36.554299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21b0812f-8ebc-48d1-ae86-0132a10f78b8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:36.590143 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:36.596687 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c249a8-3fd4-48c5-b3a6-03d230d8abcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:36.597355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c249a8-3fd4-48c5-b3a6-03d230d8abcf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:36.597972 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c249a8-3fd4-48c5-b3a6-03d230d8abcf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:36.598270 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c249a8-3fd4-48c5-b3a6-03d230d8abcf None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:36.599060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c249a8-3fd4-48c5-b3a6-03d230d8abcf None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:36.599447 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c249a8-3fd4-48c5-b3a6-03d230d8abcf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:36.600929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6c249a8-3fd4-48c5-b3a6-03d230d8abcf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:36.610696 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 78/153] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:53:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:36.616654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8b816-4669-4164-a8c0-53294b3588d7 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:36.711219 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:36.747396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8b816-4669-4164-a8c0-53294b3588d7 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:36.783645 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31a84b0d-9c96-4f73-9892-5330e0016ddc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:36.784620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a84b0d-9c96-4f73-9892-5330e0016ddc None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:36.784910 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a84b0d-9c96-4f73-9892-5330e0016ddc None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:36.785235 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31a84b0d-9c96-4f73-9892-5330e0016ddc None admin] PATH_INFO: `/v3/users/b9df4b9d4b0942339277d976457b2124` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:36.786443 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a84b0d-9c96-4f73-9892-5330e0016ddc None admin] RBAC: Authorizing `identity:get_user(user_id=b9df4b9d4b0942339277d976457b2124)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:36.787932 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31a84b0d-9c96-4f73-9892-5330e0016ddc None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:36.789890 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 76/154] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:36 2022] GET /identity/v3/users/b9df4b9d4b0942339277d976457b2124 => generated 275 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:36.795284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c249a8-3fd4-48c5-b3a6-03d230d8abcf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:36.907331 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c249a8-3fd4-48c5-b3a6-03d230d8abcf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:36.941927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbccf4f4-303d-483e-8590-b61d80de69cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:36.942590 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccf4f4-303d-483e-8590-b61d80de69cd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:36.942874 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccf4f4-303d-483e-8590-b61d80de69cd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:36.943152 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbccf4f4-303d-483e-8590-b61d80de69cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:37.003934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:37.014187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbccf4f4-303d-483e-8590-b61d80de69cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:37.015717 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbccf4f4-303d-483e-8590-b61d80de69cd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:37.026786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f351f-debf-4b54-88a9-7592ff1ce053 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:37.027077 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f351f-debf-4b54-88a9-7592ff1ce053 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:37.027378 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-825f351f-debf-4b54-88a9-7592ff1ce053 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:37.028921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 77/155] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:53:37.035324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d2105-d79a-448e-a300-dc231c17e26c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:37.035934 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d2105-d79a-448e-a300-dc231c17e26c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:37.036220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4d2105-d79a-448e-a300-dc231c17e26c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:37.056191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6c4d2105-d79a-448e-a300-dc231c17e26c None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:37.057789 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6c4d2105-d79a-448e-a300-dc231c17e26c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:37.132335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 78/156] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:53:37.159016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d2105-d79a-448e-a300-dc231c17e26c None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:37.232192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:37.259548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 79/157] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:53:37.265763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbccf4f4-303d-483e-8590-b61d80de69cd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:37.267265 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4d2105-d79a-448e-a300-dc231c17e26c None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:37.307846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431daf7-2a25-4fe9-b1f3-47f2e5c943d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:37.307846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431daf7-2a25-4fe9-b1f3-47f2e5c943d9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:37.307846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431daf7-2a25-4fe9-b1f3-47f2e5c943d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:37.307846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3431daf7-2a25-4fe9-b1f3-47f2e5c943d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:37.317831 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3431daf7-2a25-4fe9-b1f3-47f2e5c943d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:37.325741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-3431daf7-2a25-4fe9-b1f3-47f2e5c943d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:37.325741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:37.325741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:37.325741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:37.325741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:37.325741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:37.325741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:37.325741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:37.325741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:37.325741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:37.325741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:37.325741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:37.325741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:37.325741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:37.325741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:37.325741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:37.326709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:37.326709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:37.326709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:37.326709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:37.326709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:37.326709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:37.326709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:37.326709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:37.326709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:37.326709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:37.326709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:37.326709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:37.326709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:37.326709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:37.326709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:37.326709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:37.327475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:37.327475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:37.327475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:37.327475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:37.327475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:37.327475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:37.327475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:37.327475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:37.327475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:37.327475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:37.327475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:37.327475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:37.327475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 79/158] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:53:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:53:37.336749 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431daf7-2a25-4fe9-b1f3-47f2e5c943d9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:37.440392 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbccf4f4-303d-483e-8590-b61d80de69cd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:37.448033 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:37.489916 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130b9a6-ad37-4fc8-b653-c3cf89d890bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:37.490815 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130b9a6-ad37-4fc8-b653-c3cf89d890bb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:37.491096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130b9a6-ad37-4fc8-b653-c3cf89d890bb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:37.491376 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6130b9a6-ad37-4fc8-b653-c3cf89d890bb None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:37.492642 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6130b9a6-ad37-4fc8-b653-c3cf89d890bb None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:37.494131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6130b9a6-ad37-4fc8-b653-c3cf89d890bb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:37.500551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 80/159] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:37 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:37.501459 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130b9a6-ad37-4fc8-b653-c3cf89d890bb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:37.544662 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3431daf7-2a25-4fe9-b1f3-47f2e5c943d9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:37.589329 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:37.612552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5cdc3d1-e20f-4647-b1fe-9fd45a499a99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:37.613256 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdc3d1-e20f-4647-b1fe-9fd45a499a99 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:37.613544 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdc3d1-e20f-4647-b1fe-9fd45a499a99 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:37.613824 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5cdc3d1-e20f-4647-b1fe-9fd45a499a99 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:37.614596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cdc3d1-e20f-4647-b1fe-9fd45a499a99 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:37.614914 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cdc3d1-e20f-4647-b1fe-9fd45a499a99 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:37.616375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5cdc3d1-e20f-4647-b1fe-9fd45a499a99 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:37.635078 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 80/160] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:37.636527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f425fb-b5c8-42ba-9c7a-619398844aef None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:37.636820 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f425fb-b5c8-42ba-9c7a-619398844aef None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:37.637114 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f425fb-b5c8-42ba-9c7a-619398844aef None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:37.642528 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 81/161] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:37 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:37.643379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f425fb-b5c8-42ba-9c7a-619398844aef None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:37.645828 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6130b9a6-ad37-4fc8-b653-c3cf89d890bb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:37.681652 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2230db-50e6-4cf6-81b0-54ae7ed4b8dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:37.682330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2230db-50e6-4cf6-81b0-54ae7ed4b8dd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:37.682624 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2230db-50e6-4cf6-81b0-54ae7ed4b8dd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:37.682912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2230db-50e6-4cf6-81b0-54ae7ed4b8dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:37.683766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2230db-50e6-4cf6-81b0-54ae7ed4b8dd None admin] RBAC: Adding query filter params (role.id=1cc9b5d798c648c986537d006320581f, scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d, user.id=b9df4b9d4b0942339277d976457b2124) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:37.684140 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2230db-50e6-4cf6-81b0-54ae7ed4b8dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:37.685773 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2230db-50e6-4cf6-81b0-54ae7ed4b8dd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:37.692148 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 81/162] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:37 2022] GET /identity/v3/role_assignments?user.id=b9df4b9d4b0942339277d976457b2124&scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d&role.id=1cc9b5d798c648c986537d006320581f => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:37.693323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b184fb-4940-4907-9642-a374329dcd2d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:37.693618 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b184fb-4940-4907-9642-a374329dcd2d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:37.693903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b184fb-4940-4907-9642-a374329dcd2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:37.724060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-28b184fb-4940-4907-9642-a374329dcd2d None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:37.725276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-28b184fb-4940-4907-9642-a374329dcd2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:37.759404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:37.847615 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 82/163] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:37.912289 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f425fb-b5c8-42ba-9c7a-619398844aef None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:37.916188 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b184fb-4940-4907-9642-a374329dcd2d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:37.975551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174d74f-653c-4b96-94f6-26e04f1bdb56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:37.977067 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174d74f-653c-4b96-94f6-26e04f1bdb56 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:37.977687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174d74f-653c-4b96-94f6-26e04f1bdb56 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:37.978181 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b174d74f-653c-4b96-94f6-26e04f1bdb56 None admin] PATH_INFO: `/v3/users/7effefc613a44b799e8f81fc6a35b678` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:37.979859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174d74f-653c-4b96-94f6-26e04f1bdb56 None admin] RBAC: Authorizing `identity:get_user(user_id=7effefc613a44b799e8f81fc6a35b678)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:37.981605 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b174d74f-653c-4b96-94f6-26e04f1bdb56 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:37.983582 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 82/164] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:37 2022] GET /identity/v3/users/7effefc613a44b799e8f81fc6a35b678 => generated 247 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:37.990366 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174d74f-653c-4b96-94f6-26e04f1bdb56 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:38.061588 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:38.116856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b174d74f-653c-4b96-94f6-26e04f1bdb56 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:38.155265 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898bfe7-2e0f-411c-88e3-5d568344ba11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:38.156342 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898bfe7-2e0f-411c-88e3-5d568344ba11 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:38.156870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898bfe7-2e0f-411c-88e3-5d568344ba11 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:38.157434 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898bfe7-2e0f-411c-88e3-5d568344ba11 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:38.159682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b184fb-4940-4907-9642-a374329dcd2d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:38.210316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6566a1-da95-4d9f-a771-66ff344fd611 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:38.211575 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6566a1-da95-4d9f-a771-66ff344fd611 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:38.211891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6566a1-da95-4d9f-a771-66ff344fd611 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:38.212170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f6566a1-da95-4d9f-a771-66ff344fd611 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:38.229873 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f6566a1-da95-4d9f-a771-66ff344fd611 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:38.232205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898bfe7-2e0f-411c-88e3-5d568344ba11 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:38.233907 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898bfe7-2e0f-411c-88e3-5d568344ba11 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:38.241457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0f6566a1-da95-4d9f-a771-66ff344fd611 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:53:38.241457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:38.241457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:38.241457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:38.241457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:38.241457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:38.241457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:38.241457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:38.241457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:38.241457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:38.241457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:38.241457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:38.241457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:38.241457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:38.241457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:38.241457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:38.242378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:38.242378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:38.242378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:38.242378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:38.242378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:38.242378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:38.242378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:38.242378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:38.242378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:38.242378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:38.242378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:38.242378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:53:38.242378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:38.242378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:38.242378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:38.242378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:38.243403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:38.243403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:38.243403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:38.243403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:38.243403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:38.243403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:38.243403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:38.243403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:38.243403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:38.243403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:38.243403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:53:38.243403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:53:38.243403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 83/165] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:37 2022] GET /identity/v3/roles/reader => generated 84 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:38.251960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6566a1-da95-4d9f-a771-66ff344fd611 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:38.277880 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:38.347945 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:38.427383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 83/166] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:53:38.431922 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1427e70-79e9-4224-80f9-3993479562b6 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:38.432343 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1427e70-79e9-4224-80f9-3993479562b6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:38.432725 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1427e70-79e9-4224-80f9-3993479562b6 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:38.439019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 84/167] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:38.439999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1427e70-79e9-4224-80f9-3993479562b6 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:38.463235 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f6566a1-da95-4d9f-a771-66ff344fd611 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:38.524726 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6118c701-8a5c-460a-a216-a4403626a06a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:38.525415 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118c701-8a5c-460a-a216-a4403626a06a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:38.525782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118c701-8a5c-460a-a216-a4403626a06a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:38.526108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6118c701-8a5c-460a-a216-a4403626a06a None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:38.526889 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6118c701-8a5c-460a-a216-a4403626a06a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:38.527208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6118c701-8a5c-460a-a216-a4403626a06a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:38.529892 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6118c701-8a5c-460a-a216-a4403626a06a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:38.546295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 84/168] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:38 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:38.547442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ac431-7533-41d2-af4f-4be655422462 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:38.547769 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ac431-7533-41d2-af4f-4be655422462 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:38.548089 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb2ac431-7533-41d2-af4f-4be655422462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:38.574102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-cb2ac431-7533-41d2-af4f-4be655422462 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:38.575315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cb2ac431-7533-41d2-af4f-4be655422462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:38.613609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1427e70-79e9-4224-80f9-3993479562b6 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:38.670466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 85/169] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:38.672326 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32be3de-5c5c-46bb-afd9-2349ffe43585 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:38.673282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32be3de-5c5c-46bb-afd9-2349ffe43585 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:38.673572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32be3de-5c5c-46bb-afd9-2349ffe43585 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:38.673856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32be3de-5c5c-46bb-afd9-2349ffe43585 None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:38.675044 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32be3de-5c5c-46bb-afd9-2349ffe43585 None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:38.676571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32be3de-5c5c-46bb-afd9-2349ffe43585 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:38.676692 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ac431-7533-41d2-af4f-4be655422462 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:38.678666 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 85/170] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:38 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:38.679877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda915cc-9be3-4e97-9dd4-606d61f198cc None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:38.680178 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda915cc-9be3-4e97-9dd4-606d61f198cc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:38.680773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda915cc-9be3-4e97-9dd4-606d61f198cc None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:38.682013 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 86/171] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:38.687841 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda915cc-9be3-4e97-9dd4-606d61f198cc None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:38.696047 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:38.723123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:38.813608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda915cc-9be3-4e97-9dd4-606d61f198cc None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:38.814842 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb2ac431-7533-41d2-af4f-4be655422462 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:38.856210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3a286-8227-4167-a55b-2d7c684c9ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:38.857164 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3a286-8227-4167-a55b-2d7c684c9ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:38.857590 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3a286-8227-4167-a55b-2d7c684c9ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:38.858002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d3a286-8227-4167-a55b-2d7c684c9ff6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:38.858975 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3a286-8227-4167-a55b-2d7c684c9ff6 None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d, user.id=7effefc613a44b799e8f81fc6a35b678) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:38.859186 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b3e9e7-9243-4c54-aa0d-692159bbb2db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:38.859644 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3a286-8227-4167-a55b-2d7c684c9ff6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:38.860229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3e9e7-9243-4c54-aa0d-692159bbb2db None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:38.860634 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3e9e7-9243-4c54-aa0d-692159bbb2db None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:38.860984 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b3e9e7-9243-4c54-aa0d-692159bbb2db None admin] PATH_INFO: `/v3/users/50d9b6abb4e3401e883c58efc08b6eed` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:38.861787 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d3a286-8227-4167-a55b-2d7c684c9ff6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:38.862338 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3e9e7-9243-4c54-aa0d-692159bbb2db None admin] RBAC: Authorizing `identity:get_user(user_id=50d9b6abb4e3401e883c58efc08b6eed)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:38.864196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b3e9e7-9243-4c54-aa0d-692159bbb2db None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:38.870938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 86/172] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:38 2022] GET /identity/v3/users/50d9b6abb4e3401e883c58efc08b6eed => generated 289 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:38.870938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96d5a7-baa1-4b5a-8ce4-ce1d672629d5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:38.870938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96d5a7-baa1-4b5a-8ce4-ce1d672629d5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:38.870938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c96d5a7-baa1-4b5a-8ce4-ce1d672629d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:38.873973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 87/173] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:38 2022] GET /identity/v3/role_assignments?user.id=7effefc613a44b799e8f81fc6a35b678&scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:38.876030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3a286-8227-4167-a55b-2d7c684c9ff6 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:38.889001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2c96d5a7-baa1-4b5a-8ce4-ce1d672629d5 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:38.890884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2c96d5a7-baa1-4b5a-8ce4-ce1d672629d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:39.024320 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 87/174] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:39.025566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96d5a7-baa1-4b5a-8ce4-ce1d672629d5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:39.030061 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:39.037780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:39.074129 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d3a286-8227-4167-a55b-2d7c684c9ff6 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:39.119997 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80566b97-373c-48ae-9e9f-d51d9e017bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:39.120965 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80566b97-373c-48ae-9e9f-d51d9e017bad None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:39.121251 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80566b97-373c-48ae-9e9f-d51d9e017bad None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:39.121818 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80566b97-373c-48ae-9e9f-d51d9e017bad None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:39.134313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80566b97-373c-48ae-9e9f-d51d9e017bad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:39.143121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-80566b97-373c-48ae-9e9f-d51d9e017bad None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:39.143121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:39.143121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:39.143121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:39.143121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:39.143121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:39.143121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:39.143121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:39.143121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:39.143121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:39.143121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:39.143121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:39.143121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:39.143121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:39.143121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:39.143121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:39.144098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:39.144098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:39.144098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:39.144098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:39.144098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:39.144098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:39.144098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:39.144098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:39.144098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:39.144098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:39.144098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:39.144098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:39.144098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:39.144098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:39.144098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:39.144098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:39.145234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:39.145234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:39.145234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:39.145234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:39.145234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:39.145234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:39.145234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:39.145234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:39.145234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:39.145234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:39.145234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:39.145234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:39.145234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 88/175] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:53:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:39.146055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80566b97-373c-48ae-9e9f-d51d9e017bad None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:39.207448 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c96d5a7-baa1-4b5a-8ce4-ce1d672629d5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:39.243857 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa50652-5c22-44c5-9787-6850d0c8adb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:39.244932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa50652-5c22-44c5-9787-6850d0c8adb5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:39.245352 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa50652-5c22-44c5-9787-6850d0c8adb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:39.245736 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa50652-5c22-44c5-9787-6850d0c8adb5 None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:39.248122 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa50652-5c22-44c5-9787-6850d0c8adb5 None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:39.249736 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa50652-5c22-44c5-9787-6850d0c8adb5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:39.251648 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 88/176] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:39 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:39.252557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa50652-5c22-44c5-9787-6850d0c8adb5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:39.265629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80566b97-373c-48ae-9e9f-d51d9e017bad None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:39.303094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71429dec-bbf1-47d5-b1a2-8002f7a914a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:39.303094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71429dec-bbf1-47d5-b1a2-8002f7a914a3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:39.303094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71429dec-bbf1-47d5-b1a2-8002f7a914a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:39.303094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71429dec-bbf1-47d5-b1a2-8002f7a914a3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:39.314490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71429dec-bbf1-47d5-b1a2-8002f7a914a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:39.323867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-71429dec-bbf1-47d5-b1a2-8002f7a914a3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:39.323867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:39.323867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:39.323867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:39.323867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:39.323867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:39.323867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:39.323867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:39.323867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:39.323867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:39.323867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:39.323867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:39.323867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:39.323867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:39.323867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:39.323867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:39.324773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:39.324773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:39.324773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:39.324773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:39.324773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:39.324773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:39.324773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:39.324773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:39.324773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:39.324773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:39.324773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:39.324773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:39.324773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:39.324773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:39.324773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:39.324773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:39.325595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:39.325595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:39.325595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:39.325595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:39.325595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:39.325595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:39.325595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:39.325595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:39.325595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:39.325595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:39.325595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:39.325595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:39.325595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 89/177] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:39 2022] GET /identity/v3/roles/member => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:39.325595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71429dec-bbf1-47d5-b1a2-8002f7a914a3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:39.334274 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:39.343603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:39.376585 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa50652-5c22-44c5-9787-6850d0c8adb5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:39.409917 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a05d4db9-8d4a-40f4-a177-806526d6d1dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:39.410596 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d4db9-8d4a-40f4-a177-806526d6d1dc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:39.410882 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d4db9-8d4a-40f4-a177-806526d6d1dc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:39.411162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05d4db9-8d4a-40f4-a177-806526d6d1dc None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:39.411944 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d4db9-8d4a-40f4-a177-806526d6d1dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:39.412257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d4db9-8d4a-40f4-a177-806526d6d1dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:39.413661 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a05d4db9-8d4a-40f4-a177-806526d6d1dc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:39.421789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 89/178] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:39.423036 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ceead-0cf1-48ba-bed3-12bb7287b2da None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:39.423326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ceead-0cf1-48ba-bed3-12bb7287b2da None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:39.423605 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274ceead-0cf1-48ba-bed3-12bb7287b2da None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:39.426822 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 90/179] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:39.426822 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ceead-0cf1-48ba-bed3-12bb7287b2da None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:39.438934 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71429dec-bbf1-47d5-b1a2-8002f7a914a3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:39.473403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3820f96-99d8-4497-8d14-c6a7cccecf86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:39.474315 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3820f96-99d8-4497-8d14-c6a7cccecf86 None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:39.474603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3820f96-99d8-4497-8d14-c6a7cccecf86 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:39.474886 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3820f96-99d8-4497-8d14-c6a7cccecf86 None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d/users/50d9b6abb4e3401e883c58efc08b6eed/roles/b7283d3771c94f458ebfcaef5784afa0` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:39.477048 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3820f96-99d8-4497-8d14-c6a7cccecf86 None admin] RBAC: Authorizing `identity:create_grant(project_id=89a647c5dfc34a43ad12c49f8f57d99d, user_id=50d9b6abb4e3401e883c58efc08b6eed, role_id=b7283d3771c94f458ebfcaef5784afa0)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:39.478520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3820f96-99d8-4497-8d14-c6a7cccecf86 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:39.492886 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 90/180] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:53:39 2022] PUT /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d/users/50d9b6abb4e3401e883c58efc08b6eed/roles/b7283d3771c94f458ebfcaef5784afa0 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:53:39.494139 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f31354-5065-4cf6-9826-7c6c9c9cb2b3 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:39.494441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f31354-5065-4cf6-9826-7c6c9c9cb2b3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:39.494727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f31354-5065-4cf6-9826-7c6c9c9cb2b3 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:39.496151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 91/181] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:39.501781 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f31354-5065-4cf6-9826-7c6c9c9cb2b3 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:39.537526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274ceead-0cf1-48ba-bed3-12bb7287b2da None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:39.574732 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1db44c9-61d6-44bf-b4c9-c6c3d466a0ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:39.575340 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db44c9-61d6-44bf-b4c9-c6c3d466a0ee None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:39.575632 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db44c9-61d6-44bf-b4c9-c6c3d466a0ee None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:39.575950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1db44c9-61d6-44bf-b4c9-c6c3d466a0ee None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:39.576732 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db44c9-61d6-44bf-b4c9-c6c3d466a0ee None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:39.577050 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db44c9-61d6-44bf-b4c9-c6c3d466a0ee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:39.578515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1db44c9-61d6-44bf-b4c9-c6c3d466a0ee None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:39.580307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:39.589318 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 91/182] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:39 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:39.593504 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa90a0-5b0c-4114-990d-192d8c651997 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:39.594276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:39.595012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa90a0-5b0c-4114-990d-192d8c651997 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:39.595246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8baa90a0-5b0c-4114-990d-192d8c651997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:39.617022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8baa90a0-5b0c-4114-990d-192d8c651997 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:39.618277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8baa90a0-5b0c-4114-990d-192d8c651997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:39.630886 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f31354-5065-4cf6-9826-7c6c9c9cb2b3 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:39.667300 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2cc2ff-f8a6-4f34-a2d0-87ad85decc96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:39.668261 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cc2ff-f8a6-4f34-a2d0-87ad85decc96 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:39.668541 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cc2ff-f8a6-4f34-a2d0-87ad85decc96 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:39.668813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2cc2ff-f8a6-4f34-a2d0-87ad85decc96 None admin] PATH_INFO: `/v3/users/6cda4100d68a4b28aeaef2a83ad10cce` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:39.685392 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2cc2ff-f8a6-4f34-a2d0-87ad85decc96 None admin] RBAC: Authorizing `identity:get_user(user_id=6cda4100d68a4b28aeaef2a83ad10cce)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:39.686945 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2cc2ff-f8a6-4f34-a2d0-87ad85decc96 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:39.689469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 92/183] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:39 2022] GET /identity/v3/users/6cda4100d68a4b28aeaef2a83ad10cce => generated 283 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:39.690506 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23665419-bca4-47e2-982c-587f01fd2189 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:39.690783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23665419-bca4-47e2-982c-587f01fd2189 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:39.691079 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23665419-bca4-47e2-982c-587f01fd2189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:39.706437 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 92/184] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:39.708256 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-23665419-bca4-47e2-982c-587f01fd2189 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:39.708795 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa90a0-5b0c-4114-990d-192d8c651997 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:39.709480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-23665419-bca4-47e2-982c-587f01fd2189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:39.787862 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 93/185] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:39.787862 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23665419-bca4-47e2-982c-587f01fd2189 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:39.823782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8baa90a0-5b0c-4114-990d-192d8c651997 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:39.850467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:39.851861 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:39.861760 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3311c81-87be-4873-8aea-c5c992b956e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:39.862656 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3311c81-87be-4873-8aea-c5c992b956e2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:39.862944 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3311c81-87be-4873-8aea-c5c992b956e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:39.863227 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3311c81-87be-4873-8aea-c5c992b956e2 None admin] PATH_INFO: `/v3/users/b9df4b9d4b0942339277d976457b2124` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:39.864452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3311c81-87be-4873-8aea-c5c992b956e2 None admin] RBAC: Authorizing `identity:get_user(user_id=b9df4b9d4b0942339277d976457b2124)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:39.865924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3311c81-87be-4873-8aea-c5c992b956e2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:39.872200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 93/186] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:39 2022] GET /identity/v3/users/b9df4b9d4b0942339277d976457b2124 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:39.872200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3311c81-87be-4873-8aea-c5c992b956e2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:39.896235 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23665419-bca4-47e2-982c-587f01fd2189 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:39.932714 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f10d6-fe0b-472b-a29e-b195e5cf6836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:39.933376 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f10d6-fe0b-472b-a29e-b195e5cf6836 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:39.933756 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f10d6-fe0b-472b-a29e-b195e5cf6836 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:39.933951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f10d6-fe0b-472b-a29e-b195e5cf6836 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:39.972426 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3311c81-87be-4873-8aea-c5c992b956e2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:40.007674 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09e347-4874-4103-a770-361cbbb90712 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:40.008544 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09e347-4874-4103-a770-361cbbb90712 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:40.008829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09e347-4874-4103-a770-361cbbb90712 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:40.009137 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad09e347-4874-4103-a770-361cbbb90712 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:40.020160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad09e347-4874-4103-a770-361cbbb90712 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:40.020506 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f10d6-fe0b-472b-a29e-b195e5cf6836 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:40.022425 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f10d6-fe0b-472b-a29e-b195e5cf6836 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:40.028969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ad09e347-4874-4103-a770-361cbbb90712 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:53:40.028969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:40.028969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:40.028969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:40.028969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:40.028969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:40.028969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:40.028969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:40.028969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:40.028969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:40.028969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:40.028969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:40.028969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:40.028969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:40.028969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:40.028969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:40.029855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:40.029855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:40.029855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:40.029855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:40.029855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:40.029855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:40.029855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:40.029855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:40.029855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:40.029855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:40.029855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:40.029855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:53:40.029855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:40.029855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:40.029855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:40.029855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:40.030670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:40.030670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:40.030670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:40.030670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:40.030670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:40.030670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:40.030670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:40.030670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:40.030670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:40.030670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:40.030670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:53:40.030670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:53:40.030670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 94/187] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:53:39 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:40.031348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09e347-4874-4103-a770-361cbbb90712 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:40.088535 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:40.096517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:40.135432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 94/188] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:53:40.136340 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f10d6-fe0b-472b-a29e-b195e5cf6836 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:40.155432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad09e347-4874-4103-a770-361cbbb90712 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:40.193149 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395bc15-b34b-4c5d-8174-a431c75231a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:40.193967 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395bc15-b34b-4c5d-8174-a431c75231a2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:40.194256 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395bc15-b34b-4c5d-8174-a431c75231a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:40.194540 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e395bc15-b34b-4c5d-8174-a431c75231a2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:40.195984 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395bc15-b34b-4c5d-8174-a431c75231a2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:40.197456 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e395bc15-b34b-4c5d-8174-a431c75231a2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:40.203999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 95/189] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:53:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:40.204969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395bc15-b34b-4c5d-8174-a431c75231a2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:40.237609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f10d6-fe0b-472b-a29e-b195e5cf6836 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:40.272281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5c4ad-b122-4f57-ab18-b4891b504ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:40.272947 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5c4ad-b122-4f57-ab18-b4891b504ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:40.273242 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5c4ad-b122-4f57-ab18-b4891b504ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:40.273526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc5c4ad-b122-4f57-ab18-b4891b504ee6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:40.303055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:40.318174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e395bc15-b34b-4c5d-8174-a431c75231a2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:40.343075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:40.348641 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5c4ad-b122-4f57-ab18-b4891b504ee6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:40.350188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc5c4ad-b122-4f57-ab18-b4891b504ee6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:40.360936 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff225c6-a36a-4d6c-aba0-63bbddda4602 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:40.361933 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff225c6-a36a-4d6c-aba0-63bbddda4602 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:40.362383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff225c6-a36a-4d6c-aba0-63bbddda4602 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:40.362679 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff225c6-a36a-4d6c-aba0-63bbddda4602 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:40.363451 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff225c6-a36a-4d6c-aba0-63bbddda4602 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:40.363773 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff225c6-a36a-4d6c-aba0-63bbddda4602 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:40.365210 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff225c6-a36a-4d6c-aba0-63bbddda4602 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:40.373696 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 96/190] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:53:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:40.374392 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff225c6-a36a-4d6c-aba0-63bbddda4602 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:40.468797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 95/191] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:53:40.469888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5c4ad-b122-4f57-ab18-b4891b504ee6 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:40.476445 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff225c6-a36a-4d6c-aba0-63bbddda4602 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:40.512944 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c8ac7-65b7-4d7a-b187-b01393911542 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:40.514104 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c8ac7-65b7-4d7a-b187-b01393911542 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:40.514396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c8ac7-65b7-4d7a-b187-b01393911542 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:40.514678 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487c8ac7-65b7-4d7a-b187-b01393911542 None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:40.515913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c8ac7-65b7-4d7a-b187-b01393911542 None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:40.517443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487c8ac7-65b7-4d7a-b187-b01393911542 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:40.519361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 97/192] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:40 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:40.520059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c8ac7-65b7-4d7a-b187-b01393911542 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:40.540443 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:40.583076 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:40.583290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc5c4ad-b122-4f57-ab18-b4891b504ee6 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:40.620956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373f379-c701-4ab1-bacc-2e0d40863d49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:40.621647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373f379-c701-4ab1-bacc-2e0d40863d49 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:40.621938 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373f379-c701-4ab1-bacc-2e0d40863d49 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:40.622221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4373f379-c701-4ab1-bacc-2e0d40863d49 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:40.622938 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4373f379-c701-4ab1-bacc-2e0d40863d49 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:40.624471 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4373f379-c701-4ab1-bacc-2e0d40863d49 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:40.638608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487c8ac7-65b7-4d7a-b187-b01393911542 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:40.653402 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 96/193] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:53:40 2022] POST /identity/v3/users => generated 297 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:53:40.654174 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373f379-c701-4ab1-bacc-2e0d40863d49 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:40.690793 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08238b0b-6b92-4f28-8a81-2d4810ae9073 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:40.691923 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08238b0b-6b92-4f28-8a81-2d4810ae9073 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:40.692345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08238b0b-6b92-4f28-8a81-2d4810ae9073 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:40.692718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08238b0b-6b92-4f28-8a81-2d4810ae9073 None admin] PATH_INFO: `/v3/users/6cda4100d68a4b28aeaef2a83ad10cce` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:40.694184 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08238b0b-6b92-4f28-8a81-2d4810ae9073 None admin] RBAC: Authorizing `identity:get_user(user_id=6cda4100d68a4b28aeaef2a83ad10cce)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:40.695836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08238b0b-6b92-4f28-8a81-2d4810ae9073 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:40.697707 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 98/194] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:40 2022] GET /identity/v3/users/6cda4100d68a4b28aeaef2a83ad10cce => generated 283 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:40.702942 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08238b0b-6b92-4f28-8a81-2d4810ae9073 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:40.819190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08238b0b-6b92-4f28-8a81-2d4810ae9073 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:40.819770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4373f379-c701-4ab1-bacc-2e0d40863d49 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:40.842595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:40.844817 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:40.864111 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c1211-3511-4b5c-bc67-a17ba7ef5c6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:40.864883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c1211-3511-4b5c-bc67-a17ba7ef5c6d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:40.865170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c1211-3511-4b5c-bc67-a17ba7ef5c6d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:40.865452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7c1211-3511-4b5c-bc67-a17ba7ef5c6d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:40.866297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c1211-3511-4b5c-bc67-a17ba7ef5c6d None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7, user.id=6cda4100d68a4b28aeaef2a83ad10cce) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:40.866612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c1211-3511-4b5c-bc67-a17ba7ef5c6d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:40.868735 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7c1211-3511-4b5c-bc67-a17ba7ef5c6d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:40.871618 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1728af-69d1-4558-a275-da21b92846c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:40.872548 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1728af-69d1-4558-a275-da21b92846c0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:40.872843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1728af-69d1-4558-a275-da21b92846c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:40.873125 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c1728af-69d1-4558-a275-da21b92846c0 None admin] PATH_INFO: `/v3/projects/e987fcc8b09c46a7a738a5bc966cfe63` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:40.874334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1728af-69d1-4558-a275-da21b92846c0 None admin] RBAC: Authorizing `identity:get_project(project_id=e987fcc8b09c46a7a738a5bc966cfe63)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:40.875071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 99/195] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:40 2022] GET /identity/v3/role_assignments?user.id=6cda4100d68a4b28aeaef2a83ad10cce&scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 259 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:40.875895 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c1728af-69d1-4558-a275-da21b92846c0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:40.876279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c1211-3511-4b5c-bc67-a17ba7ef5c6d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:40.877621 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 97/196] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:40 2022] GET /identity/v3/projects/e987fcc8b09c46a7a738a5bc966cfe63 => generated 312 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:40.883873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1728af-69d1-4558-a275-da21b92846c0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:41.016220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c1728af-69d1-4558-a275-da21b92846c0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:41.017769 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7c1211-3511-4b5c-bc67-a17ba7ef5c6d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:41.059071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9e61a-0fb6-447e-96aa-932605d7d734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:41.059763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9e61a-0fb6-447e-96aa-932605d7d734 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:41.060135 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9e61a-0fb6-447e-96aa-932605d7d734 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:41.060424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a9e61a-0fb6-447e-96aa-932605d7d734 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:41.061680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314c36e-62b7-4264-a509-272575443895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:41.062375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314c36e-62b7-4264-a509-272575443895 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:41.062662 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314c36e-62b7-4264-a509-272575443895 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:41.062950 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9314c36e-62b7-4264-a509-272575443895 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:41.063874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314c36e-62b7-4264-a509-272575443895 None admin] RBAC: Adding query filter params (role.id=9c81542451e54c0ea22a20c026be6be7, scope.project.id=e987fcc8b09c46a7a738a5bc966cfe63, user.id=b9df4b9d4b0942339277d976457b2124) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:41.064200 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314c36e-62b7-4264-a509-272575443895 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:41.065717 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9314c36e-62b7-4264-a509-272575443895 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:41.072980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 98/197] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:40 2022] GET /identity/v3/role_assignments?user.id=b9df4b9d4b0942339277d976457b2124&scope.project.id=e987fcc8b09c46a7a738a5bc966cfe63&role.id=9c81542451e54c0ea22a20c026be6be7 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:41.145605 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:41.169219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9e61a-0fb6-447e-96aa-932605d7d734 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:41.170742 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a9e61a-0fb6-447e-96aa-932605d7d734 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:41.284239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 100/198] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:53:41.289668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314c36e-62b7-4264-a509-272575443895 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:41.353801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:41.420842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9314c36e-62b7-4264-a509-272575443895 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:41.458512 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04d95b9a-b07a-4a0e-9456-a287ed6426a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:41.459407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d95b9a-b07a-4a0e-9456-a287ed6426a9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:41.459695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d95b9a-b07a-4a0e-9456-a287ed6426a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:41.460026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04d95b9a-b07a-4a0e-9456-a287ed6426a9 None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:41.461578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d95b9a-b07a-4a0e-9456-a287ed6426a9 None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:41.463053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04d95b9a-b07a-4a0e-9456-a287ed6426a9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:41.465118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 99/199] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:41 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:41.471002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9e61a-0fb6-447e-96aa-932605d7d734 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:41.581872 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:41.598271 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a9e61a-0fb6-447e-96aa-932605d7d734 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:41.638847 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd6de7a9-a52d-4eba-ad65-8c29287a1eb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:41.639515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6de7a9-a52d-4eba-ad65-8c29287a1eb1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:41.639800 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6de7a9-a52d-4eba-ad65-8c29287a1eb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:41.640102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6de7a9-a52d-4eba-ad65-8c29287a1eb1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:41.640946 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6de7a9-a52d-4eba-ad65-8c29287a1eb1 None admin] RBAC: Adding query filter params (role.id=1cc9b5d798c648c986537d006320581f, scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7, user.id=6cda4100d68a4b28aeaef2a83ad10cce) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:41.641332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6de7a9-a52d-4eba-ad65-8c29287a1eb1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:41.642817 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd6de7a9-a52d-4eba-ad65-8c29287a1eb1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:41.648964 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 101/200] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:41 2022] GET /identity/v3/role_assignments?user.id=6cda4100d68a4b28aeaef2a83ad10cce&scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7&role.id=1cc9b5d798c648c986537d006320581f => generated 259 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:41.924168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a5464-ca02-4872-a5df-44cc26f309d1 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:41.924648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a5464-ca02-4872-a5df-44cc26f309d1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:41.924648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b7a5464-ca02-4872-a5df-44cc26f309d1 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:41.926320 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 100/201] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:41.931694 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129c737-b6c1-4cec-9ca3-04343c4b14e7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:41.932024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129c737-b6c1-4cec-9ca3-04343c4b14e7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:41.932314 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129c737-b6c1-4cec-9ca3-04343c4b14e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:41.956864 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8129c737-b6c1-4cec-9ca3-04343c4b14e7 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:41.958087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8129c737-b6c1-4cec-9ca3-04343c4b14e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:42.039196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 102/202] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:42.066345 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a5464-ca02-4872-a5df-44cc26f309d1 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:42.191409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b7a5464-ca02-4872-a5df-44cc26f309d1 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:42.206274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:42.233717 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355270ed-e18e-47d3-a1ef-45455b38b393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:42.234551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355270ed-e18e-47d3-a1ef-45455b38b393 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:42.234844 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355270ed-e18e-47d3-a1ef-45455b38b393 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:42.235168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-355270ed-e18e-47d3-a1ef-45455b38b393 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:42.251163 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-355270ed-e18e-47d3-a1ef-45455b38b393 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:42.263026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-355270ed-e18e-47d3-a1ef-45455b38b393 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:53:42.263026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:42.263026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:42.263026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:42.263026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:42.263026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:42.263026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:42.263026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:42.263026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:42.263026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:42.263026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:42.263026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:42.263026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:42.263026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:42.263026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:42.263026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:42.265187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:42.265187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:42.265187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:42.265187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:42.265187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:42.265187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:42.265187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:42.265187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:42.265187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:42.265187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:42.265187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:42.265187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:42.265187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:42.265187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:42.265187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:42.265187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:42.267169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:42.267169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:42.267169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:42.267169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:42.267169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:42.267169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:42.267169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:42.267169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:42.267169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:42.267169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:42.267169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:53:42.267169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:42.267169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 101/203] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:42 2022] GET /identity/v3/roles/reader => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:42.274597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129c737-b6c1-4cec-9ca3-04343c4b14e7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:42.399319 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:42.426589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8129c737-b6c1-4cec-9ca3-04343c4b14e7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:42.476830 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95251cad-8eb2-4a57-a772-bab63dd3687a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:42.477464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95251cad-8eb2-4a57-a772-bab63dd3687a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:42.477743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95251cad-8eb2-4a57-a772-bab63dd3687a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:42.478027 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95251cad-8eb2-4a57-a772-bab63dd3687a None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:42.478799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95251cad-8eb2-4a57-a772-bab63dd3687a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:42.479121 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95251cad-8eb2-4a57-a772-bab63dd3687a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:42.480578 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95251cad-8eb2-4a57-a772-bab63dd3687a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:42.495357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 103/204] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:42 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:42.500943 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355270ed-e18e-47d3-a1ef-45455b38b393 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:42.631425 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-355270ed-e18e-47d3-a1ef-45455b38b393 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:42.664172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710cefe-cc84-420c-83ff-a65618411aca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:42.665078 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710cefe-cc84-420c-83ff-a65618411aca None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:42.665365 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710cefe-cc84-420c-83ff-a65618411aca None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:42.665646 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2710cefe-cc84-420c-83ff-a65618411aca None admin] PATH_INFO: `/v3/users/50d9b6abb4e3401e883c58efc08b6eed` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:42.666859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710cefe-cc84-420c-83ff-a65618411aca None admin] RBAC: Authorizing `identity:get_user(user_id=50d9b6abb4e3401e883c58efc08b6eed)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:42.668375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2710cefe-cc84-420c-83ff-a65618411aca None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:42.670384 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 102/205] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:42 2022] GET /identity/v3/users/50d9b6abb4e3401e883c58efc08b6eed => generated 289 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:42.675728 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95251cad-8eb2-4a57-a772-bab63dd3687a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:42.834770 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95251cad-8eb2-4a57-a772-bab63dd3687a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:42.876452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a620b0d-4a28-4f0a-970f-0958eecafb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:42.877121 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a620b0d-4a28-4f0a-970f-0958eecafb4e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:42.877407 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a620b0d-4a28-4f0a-970f-0958eecafb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:42.877689 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a620b0d-4a28-4f0a-970f-0958eecafb4e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:42.893538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:42.971289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a620b0d-4a28-4f0a-970f-0958eecafb4e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:42.973169 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a620b0d-4a28-4f0a-970f-0958eecafb4e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:43.101530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 104/206] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:53:43.107207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710cefe-cc84-420c-83ff-a65618411aca None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:43.195877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:43.275842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2710cefe-cc84-420c-83ff-a65618411aca None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:43.338788 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2e0d5d-d3dc-4474-9590-4ddc281c3701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:43.340119 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e0d5d-d3dc-4474-9590-4ddc281c3701 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:43.340613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e0d5d-d3dc-4474-9590-4ddc281c3701 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:43.341088 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2e0d5d-d3dc-4474-9590-4ddc281c3701 None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:43.345293 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e0d5d-d3dc-4474-9590-4ddc281c3701 None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:43.347008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b2e0d5d-d3dc-4474-9590-4ddc281c3701 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:43.349355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 103/207] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:43 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:43.354911 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a620b0d-4a28-4f0a-970f-0958eecafb4e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:43.407282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:43.489646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a620b0d-4a28-4f0a-970f-0958eecafb4e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:43.546987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f63f750b-2d4b-47de-90f2-f61fdafd6fd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:43.548130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f750b-2d4b-47de-90f2-f61fdafd6fd6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:43.548654 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f750b-2d4b-47de-90f2-f61fdafd6fd6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:43.549136 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f63f750b-2d4b-47de-90f2-f61fdafd6fd6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:43.550209 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f750b-2d4b-47de-90f2-f61fdafd6fd6 None admin] RBAC: Adding query filter params (role.id=b7283d3771c94f458ebfcaef5784afa0, scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d, user.id=50d9b6abb4e3401e883c58efc08b6eed) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:43.550734 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f750b-2d4b-47de-90f2-f61fdafd6fd6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:43.552499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f63f750b-2d4b-47de-90f2-f61fdafd6fd6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:43.564645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 105/208] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:43 2022] GET /identity/v3/role_assignments?user.id=50d9b6abb4e3401e883c58efc08b6eed&scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d&role.id=b7283d3771c94f458ebfcaef5784afa0 => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:43.673620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48cc80b-ab5c-42d9-8861-d74d32254841 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:43.674289 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48cc80b-ab5c-42d9-8861-d74d32254841 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:43.674773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48cc80b-ab5c-42d9-8861-d74d32254841 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:43.676495 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 104/209] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:43.682382 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6942367-6902-437c-b552-64d1d3b28854 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:43.682970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6942367-6902-437c-b552-64d1d3b28854 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:43.683451 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6942367-6902-437c-b552-64d1d3b28854 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:43.704982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e6942367-6902-437c-b552-64d1d3b28854 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:43.706572 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e6942367-6902-437c-b552-64d1d3b28854 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:43.729455 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7456b5-f3df-4b9f-8cf4-d031e5bc5ce4 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:43.730116 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7456b5-f3df-4b9f-8cf4-d031e5bc5ce4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:43.730601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7456b5-f3df-4b9f-8cf4-d031e5bc5ce4 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:43.732249 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 105/210] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:43.745655 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfd50d-515b-4b67-a62c-f49b8583f776 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:43.746320 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfd50d-515b-4b67-a62c-f49b8583f776 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:43.746804 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abfd50d-515b-4b67-a62c-f49b8583f776 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:43.779570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8abfd50d-515b-4b67-a62c-f49b8583f776 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:43.780837 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8abfd50d-515b-4b67-a62c-f49b8583f776 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:43.827058 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 106/211] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:43.856772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6942367-6902-437c-b552-64d1d3b28854 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:43.886670 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 106/212] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) May 25 15:53:43.908818 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:43.923885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfd50d-515b-4b67-a62c-f49b8583f776 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:43.972190 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:43.988133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6942367-6902-437c-b552-64d1d3b28854 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:44.033631 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1a8ca-3f1e-45f9-bc55-6b4676af5cab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:44.034692 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1a8ca-3f1e-45f9-bc55-6b4676af5cab None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:44.035117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1a8ca-3f1e-45f9-bc55-6b4676af5cab None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:44.035533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6f1a8ca-3f1e-45f9-bc55-6b4676af5cab None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:44.038924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abfd50d-515b-4b67-a62c-f49b8583f776 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:44.048241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6f1a8ca-3f1e-45f9-bc55-6b4676af5cab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:44.057063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f6f1a8ca-3f1e-45f9-bc55-6b4676af5cab None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:44.057063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:44.057063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:44.057063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:44.057063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:44.057063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:44.057063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:44.057063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:44.057063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:44.057063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:44.057063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:44.057063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:44.057063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:44.057063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:44.057063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:44.057063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:44.058196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:44.058196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:44.058196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:44.058196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:44.058196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:44.058196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:44.058196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:44.058196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:44.058196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:44.058196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:44.058196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:44.058196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:53:44.058196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:44.058196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:44.058196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:44.058196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:44.059190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:44.059190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:44.059190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:44.059190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:44.059190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:44.059190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:44.059190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:44.059190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:44.059190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:44.059190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:44.059190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:44.059190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:53:44.061285 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 107/213] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:53:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:53:44.071759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1a8ca-3f1e-45f9-bc55-6b4676af5cab None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:44.083184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097a238-0a09-49c8-a6d2-10214f86dcf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:44.084360 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097a238-0a09-49c8-a6d2-10214f86dcf3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:44.084826 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097a238-0a09-49c8-a6d2-10214f86dcf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:44.085277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3097a238-0a09-49c8-a6d2-10214f86dcf3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:44.087515 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097a238-0a09-49c8-a6d2-10214f86dcf3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:44.089305 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3097a238-0a09-49c8-a6d2-10214f86dcf3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:44.091344 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 107/214] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:53:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:44.096918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097a238-0a09-49c8-a6d2-10214f86dcf3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:44.185462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:44.192852 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6f1a8ca-3f1e-45f9-bc55-6b4676af5cab None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:44.204385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3097a238-0a09-49c8-a6d2-10214f86dcf3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:44.214179 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:44.233238 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0dabcb5-4167-444d-9870-d924812d3189 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:44.233945 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dabcb5-4167-444d-9870-d924812d3189 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:44.234235 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dabcb5-4167-444d-9870-d924812d3189 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:44.234514 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0dabcb5-4167-444d-9870-d924812d3189 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:44.235371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dabcb5-4167-444d-9870-d924812d3189 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:44.235691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dabcb5-4167-444d-9870-d924812d3189 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:44.237205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0dabcb5-4167-444d-9870-d924812d3189 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:44.243240 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0cd95a5-1ac0-4e82-a604-a66e950fa5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:44.243980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd95a5-1ac0-4e82-a604-a66e950fa5fb None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:44.244265 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd95a5-1ac0-4e82-a604-a66e950fa5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:44.244540 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0cd95a5-1ac0-4e82-a604-a66e950fa5fb None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:44.245271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd95a5-1ac0-4e82-a604-a66e950fa5fb None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:44.246771 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0cd95a5-1ac0-4e82-a604-a66e950fa5fb None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:44.251961 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 108/215] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:44.253279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99fdc3-fa35-434d-b6e7-867d2d9fd3ff None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:44.253578 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99fdc3-fa35-434d-b6e7-867d2d9fd3ff None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:44.253957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe99fdc3-fa35-434d-b6e7-867d2d9fd3ff None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:44.255225 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 109/216] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:44.255991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99fdc3-fa35-434d-b6e7-867d2d9fd3ff None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:44.282523 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 108/217] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:53:44 2022] POST /identity/v3/users => generated 297 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:53:44.282523 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95431351-5f0e-4275-b866-bf04c4a426f8 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:44.282523 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95431351-5f0e-4275-b866-bf04c4a426f8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:44.282523 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95431351-5f0e-4275-b866-bf04c4a426f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:44.302235 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-95431351-5f0e-4275-b866-bf04c4a426f8 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:44.302235 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-95431351-5f0e-4275-b866-bf04c4a426f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:44.397924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe99fdc3-fa35-434d-b6e7-867d2d9fd3ff None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:44.400988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 109/218] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:44.417515 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d8d4b-f4a3-4aad-825b-c44a2f22633c None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:44.417804 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d8d4b-f4a3-4aad-825b-c44a2f22633c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:44.418093 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9d8d4b-f4a3-4aad-825b-c44a2f22633c None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:44.419510 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 110/219] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:44.425212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5215b56-4e13-4f32-bc0c-82f9f7c172f9 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:44.425501 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5215b56-4e13-4f32-bc0c-82f9f7c172f9 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:44.425777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5215b56-4e13-4f32-bc0c-82f9f7c172f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:44.441284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdcd623-2d13-4ee8-804a-7b8bd9fa3b50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:44.442099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdcd623-2d13-4ee8-804a-7b8bd9fa3b50 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:44.442391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdcd623-2d13-4ee8-804a-7b8bd9fa3b50 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:44.442658 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afdcd623-2d13-4ee8-804a-7b8bd9fa3b50 None admin] PATH_INFO: `/v3/users/6cda4100d68a4b28aeaef2a83ad10cce` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:44.443935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdcd623-2d13-4ee8-804a-7b8bd9fa3b50 None admin] RBAC: Authorizing `identity:get_user(user_id=6cda4100d68a4b28aeaef2a83ad10cce)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:44.444253 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f5215b56-4e13-4f32-bc0c-82f9f7c172f9 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:44.445440 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afdcd623-2d13-4ee8-804a-7b8bd9fa3b50 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:44.445548 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f5215b56-4e13-4f32-bc0c-82f9f7c172f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:44.447204 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 110/220] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:44 2022] GET /identity/v3/users/6cda4100d68a4b28aeaef2a83ad10cce => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:44.452785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdcd623-2d13-4ee8-804a-7b8bd9fa3b50 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:44.480533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:44.523722 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 111/221] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:53:44.524624 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5215b56-4e13-4f32-bc0c-82f9f7c172f9 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:44.559656 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:44.574728 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afdcd623-2d13-4ee8-804a-7b8bd9fa3b50 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:44.620834 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c23c62-3036-4d1b-aa06-5d085736ba0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:44.622457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c23c62-3036-4d1b-aa06-5d085736ba0f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:44.622756 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c23c62-3036-4d1b-aa06-5d085736ba0f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:44.623042 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c23c62-3036-4d1b-aa06-5d085736ba0f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:44.638565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c23c62-3036-4d1b-aa06-5d085736ba0f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:44.646559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-29c23c62-3036-4d1b-aa06-5d085736ba0f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:44.646559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:44.646559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:44.646559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:44.646559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:44.646559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:44.646559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:44.646559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:44.646559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:44.646559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:44.646559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:44.646559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:44.646559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:44.646559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:44.646559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:44.646559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:44.647496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:44.647496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:44.647496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:44.647496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:44.647496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:44.647496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:44.647496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:44.647496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:44.647496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:44.647496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:44.647496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:44.647496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:53:44.647496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:44.647496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:44.647496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:44.647496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:44.648317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:44.648317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:44.648317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:44.648317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:44.648317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:44.648317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:44.648317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:44.648317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:44.648317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:44.648317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:44.648317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:44.648317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:53:44.648317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 111/222] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:44.648955 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c23c62-3036-4d1b-aa06-5d085736ba0f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:44.665342 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5215b56-4e13-4f32-bc0c-82f9f7c172f9 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:44.708584 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e22dc-3fa0-4f50-a428-eac6b08fa26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:44.709841 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e22dc-3fa0-4f50-a428-eac6b08fa26b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:44.710297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e22dc-3fa0-4f50-a428-eac6b08fa26b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:44.710740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a0e22dc-3fa0-4f50-a428-eac6b08fa26b None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:44.712631 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e22dc-3fa0-4f50-a428-eac6b08fa26b None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:44.714362 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a0e22dc-3fa0-4f50-a428-eac6b08fa26b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:44.716852 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 112/223] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:44 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:44.717871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e22dc-3fa0-4f50-a428-eac6b08fa26b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:44.744985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:44.760848 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c23c62-3036-4d1b-aa06-5d085736ba0f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:44.795582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24eb0b-424b-448e-8256-0204fc8203e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:44.796444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24eb0b-424b-448e-8256-0204fc8203e8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:44.796735 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24eb0b-424b-448e-8256-0204fc8203e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:44.797016 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df24eb0b-424b-448e-8256-0204fc8203e8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:44.807635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df24eb0b-424b-448e-8256-0204fc8203e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:44.815983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-df24eb0b-424b-448e-8256-0204fc8203e8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:53:44.815983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:44.815983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:44.815983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:44.815983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:44.815983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:44.815983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:44.815983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:44.815983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:44.815983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:44.815983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:44.815983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:44.815983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:44.815983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:44.815983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:44.815983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:44.816914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:44.816914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:44.816914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:44.816914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:44.816914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:44.816914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:44.816914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:44.816914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:44.816914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:44.816914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:44.816914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:44.816914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:53:44.816914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:44.816914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:44.816914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:44.816914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:44.817698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:44.817698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:44.817698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:44.817698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:44.817698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:44.817698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:44.817698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:44.817698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:44.817698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:44.817698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:44.817698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:53:44.817698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:53:44.817698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 112/224] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:53:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:44.818335 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24eb0b-424b-448e-8256-0204fc8203e8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:44.824168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:44.838495 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a0e22dc-3fa0-4f50-a428-eac6b08fa26b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:44.874911 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240fc3ce-fa0f-4115-aa32-edf2a5f0557a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:44.876057 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fc3ce-fa0f-4115-aa32-edf2a5f0557a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:44.876537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fc3ce-fa0f-4115-aa32-edf2a5f0557a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:44.877001 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fc3ce-fa0f-4115-aa32-edf2a5f0557a None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:44.877944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240fc3ce-fa0f-4115-aa32-edf2a5f0557a None admin] RBAC: Adding query filter params (name=member) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:44.878465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240fc3ce-fa0f-4115-aa32-edf2a5f0557a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:44.880155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-240fc3ce-fa0f-4115-aa32-edf2a5f0557a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:44.889335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 113/225] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:44 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:44.894198 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240fc3ce-fa0f-4115-aa32-edf2a5f0557a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:44.924975 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df24eb0b-424b-448e-8256-0204fc8203e8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:44.960075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f302a-4f75-4d0a-a0f8-f62e2be826a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:44.961513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f302a-4f75-4d0a-a0f8-f62e2be826a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:44.961805 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f302a-4f75-4d0a-a0f8-f62e2be826a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:44.962089 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4f302a-4f75-4d0a-a0f8-f62e2be826a3 None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7/users/6cda4100d68a4b28aeaef2a83ad10cce/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:44.964115 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f302a-4f75-4d0a-a0f8-f62e2be826a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8be03eb22fb4d4d9fb156c025f259d7, user_id=6cda4100d68a4b28aeaef2a83ad10cce, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:44.966251 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4f302a-4f75-4d0a-a0f8-f62e2be826a3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:44.986809 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 113/226] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:53:44 2022] PUT /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7/users/6cda4100d68a4b28aeaef2a83ad10cce/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:53:44.987707 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f302a-4f75-4d0a-a0f8-f62e2be826a3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:45.004654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240fc3ce-fa0f-4115-aa32-edf2a5f0557a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:45.019781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:45.077383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ce0fa-c8d3-4ccd-a33d-752b448caaba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:45.078148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ce0fa-c8d3-4ccd-a33d-752b448caaba None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:45.078437 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ce0fa-c8d3-4ccd-a33d-752b448caaba None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:45.078758 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186ce0fa-c8d3-4ccd-a33d-752b448caaba None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:45.079614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ce0fa-c8d3-4ccd-a33d-752b448caaba None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:45.080050 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ce0fa-c8d3-4ccd-a33d-752b448caaba None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:45.081478 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-186ce0fa-c8d3-4ccd-a33d-752b448caaba None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:45.099473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 114/227] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:53:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:45.105381 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ce0fa-c8d3-4ccd-a33d-752b448caaba None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:45.122939 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4f302a-4f75-4d0a-a0f8-f62e2be826a3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:45.140041 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:45.163016 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003f183-564e-4cac-8494-671581d386cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:45.164007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003f183-564e-4cac-8494-671581d386cc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:45.164521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003f183-564e-4cac-8494-671581d386cc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:45.164809 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6003f183-564e-4cac-8494-671581d386cc None admin] PATH_INFO: `/v3/users/fafa43aa9cb04a5aa9b5259fa1f93e17` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:45.200118 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003f183-564e-4cac-8494-671581d386cc None admin] RBAC: Authorizing `identity:get_user(user_id=fafa43aa9cb04a5aa9b5259fa1f93e17)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:45.200840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6003f183-564e-4cac-8494-671581d386cc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:45.202895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 114/228] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:44 2022] GET /identity/v3/users/fafa43aa9cb04a5aa9b5259fa1f93e17 => generated 297 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:45.208924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003f183-564e-4cac-8494-671581d386cc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:45.271593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186ce0fa-c8d3-4ccd-a33d-752b448caaba None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:45.291006 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:45.309314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022506cf-585b-4673-80d0-d71da1d9096e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:45.310238 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022506cf-585b-4673-80d0-d71da1d9096e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:45.310528 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022506cf-585b-4673-80d0-d71da1d9096e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:45.310813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022506cf-585b-4673-80d0-d71da1d9096e None admin] PATH_INFO: `/v3/users/6cda4100d68a4b28aeaef2a83ad10cce` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:45.312086 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022506cf-585b-4673-80d0-d71da1d9096e None admin] RBAC: Authorizing `identity:get_user(user_id=6cda4100d68a4b28aeaef2a83ad10cce)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:45.313665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022506cf-585b-4673-80d0-d71da1d9096e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:45.315725 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 115/229] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:45 2022] GET /identity/v3/users/6cda4100d68a4b28aeaef2a83ad10cce => generated 283 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:45.320629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022506cf-585b-4673-80d0-d71da1d9096e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:45.331411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6003f183-564e-4cac-8494-671581d386cc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:45.367547 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2c225-8813-4158-8ab7-cdae632f3331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:45.368242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2c225-8813-4158-8ab7-cdae632f3331 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:45.368535 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2c225-8813-4158-8ab7-cdae632f3331 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:45.368818 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2c225-8813-4158-8ab7-cdae632f3331 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:45.436824 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2c225-8813-4158-8ab7-cdae632f3331 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:45.438375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2c225-8813-4158-8ab7-cdae632f3331 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:45.484251 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:45.495980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022506cf-585b-4673-80d0-d71da1d9096e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:45.498074 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:45.538067 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 115/230] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:53:45.544146 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2c225-8813-4158-8ab7-cdae632f3331 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:45.566974 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66411268-3c23-46d4-9d3c-641a46c8d206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:45.567897 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66411268-3c23-46d4-9d3c-641a46c8d206 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:45.568276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66411268-3c23-46d4-9d3c-641a46c8d206 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:45.568563 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66411268-3c23-46d4-9d3c-641a46c8d206 None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:45.571944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66411268-3c23-46d4-9d3c-641a46c8d206 None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:45.573614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66411268-3c23-46d4-9d3c-641a46c8d206 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:45.575565 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 116/231] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:45 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:45.581217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66411268-3c23-46d4-9d3c-641a46c8d206 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:45.661141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2c225-8813-4158-8ab7-cdae632f3331 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:45.704739 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66411268-3c23-46d4-9d3c-641a46c8d206 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:45.732467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9b51e-5c73-4efe-8bf2-7ea9cd96fc1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:45.733384 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9b51e-5c73-4efe-8bf2-7ea9cd96fc1e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:45.733674 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9b51e-5c73-4efe-8bf2-7ea9cd96fc1e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:45.733957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad9b51e-5c73-4efe-8bf2-7ea9cd96fc1e None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:45.735381 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9b51e-5c73-4efe-8bf2-7ea9cd96fc1e None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:45.736913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad9b51e-5c73-4efe-8bf2-7ea9cd96fc1e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:45.738951 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 116/232] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:45 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:45.745228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9b51e-5c73-4efe-8bf2-7ea9cd96fc1e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:45.762756 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739db04e-1f9b-4eea-a4f1-de3138fe147f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:45.763794 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739db04e-1f9b-4eea-a4f1-de3138fe147f None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:45.764128 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739db04e-1f9b-4eea-a4f1-de3138fe147f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:45.764475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739db04e-1f9b-4eea-a4f1-de3138fe147f None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7/users/6cda4100d68a4b28aeaef2a83ad10cce/roles/1cc9b5d798c648c986537d006320581f` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:45.770210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739db04e-1f9b-4eea-a4f1-de3138fe147f None admin] RBAC: Authorizing `identity:create_grant(project_id=a8be03eb22fb4d4d9fb156c025f259d7, user_id=6cda4100d68a4b28aeaef2a83ad10cce, role_id=1cc9b5d798c648c986537d006320581f)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:45.771716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739db04e-1f9b-4eea-a4f1-de3138fe147f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:45.781877 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:45.807780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 117/233] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:53:45 2022] PUT /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7/users/6cda4100d68a4b28aeaef2a83ad10cce/roles/1cc9b5d798c648c986537d006320581f => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:53:45.853345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad9b51e-5c73-4efe-8bf2-7ea9cd96fc1e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:45.892361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd529d7-0a2e-43e5-9a03-b339756cc763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:45.893040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd529d7-0a2e-43e5-9a03-b339756cc763 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:45.893329 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd529d7-0a2e-43e5-9a03-b339756cc763 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:45.893612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd529d7-0a2e-43e5-9a03-b339756cc763 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:45.894474 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd529d7-0a2e-43e5-9a03-b339756cc763 None admin] RBAC: Adding query filter params (role.id=9c81542451e54c0ea22a20c026be6be7, scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7, user.id=fafa43aa9cb04a5aa9b5259fa1f93e17) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:45.894791 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd529d7-0a2e-43e5-9a03-b339756cc763 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:45.896343 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd529d7-0a2e-43e5-9a03-b339756cc763 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:45.907335 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 117/234] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:45 2022] GET /identity/v3/role_assignments?user.id=fafa43aa9cb04a5aa9b5259fa1f93e17&scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7&role.id=9c81542451e54c0ea22a20c026be6be7 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:46.861679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7170f-a8c6-43fc-8d71-332bf724b255 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:46.862215 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7170f-a8c6-43fc-8d71-332bf724b255 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:46.862292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c7170f-a8c6-43fc-8d71-332bf724b255 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:46.864066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 118/235] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:46.870018 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0394254-e076-4cd6-a755-676256191a5e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:46.870524 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0394254-e076-4cd6-a755-676256191a5e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:46.870720 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0394254-e076-4cd6-a755-676256191a5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:46.892992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c0394254-e076-4cd6-a755-676256191a5e None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:46.894205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c0394254-e076-4cd6-a755-676256191a5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:46.962907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 118/236] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:46.985918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c7170f-a8c6-43fc-8d71-332bf724b255 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:47.001579 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:47.075761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2790a-ae7f-4699-8945-df7a8b9573ba None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:47.076110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2790a-ae7f-4699-8945-df7a8b9573ba None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:47.076351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1a2790a-ae7f-4699-8945-df7a8b9573ba None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:47.077737 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 119/237] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:47.086567 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b93d5c-c9c4-4808-99ce-160d467a27eb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:47.086567 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b93d5c-c9c4-4808-99ce-160d467a27eb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:47.086567 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b93d5c-c9c4-4808-99ce-160d467a27eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:47.111087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-49b93d5c-c9c4-4808-99ce-160d467a27eb None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:47.113801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-49b93d5c-c9c4-4808-99ce-160d467a27eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:47.135774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c7170f-a8c6-43fc-8d71-332bf724b255 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:47.153898 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:47.189077 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e01cbf-166a-433f-9e07-89aa3a6bf461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:47.190258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e01cbf-166a-433f-9e07-89aa3a6bf461 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:47.190769 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e01cbf-166a-433f-9e07-89aa3a6bf461 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:47.191257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e01cbf-166a-433f-9e07-89aa3a6bf461 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:47.213359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e01cbf-166a-433f-9e07-89aa3a6bf461 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:47.215752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 120/238] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:47.233967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-59e01cbf-166a-433f-9e07-89aa3a6bf461 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:53:47.233967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:47.233967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:47.233967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:47.233967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:47.233967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:47.233967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:47.233967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:47.233967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:47.233967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:47.233967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:47.233967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:47.233967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:47.233967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:47.233967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:47.233967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:47.235148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:47.235148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:47.235148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:47.235148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:47.235148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:47.235148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:47.235148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:47.235148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:47.235148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:47.235148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:47.235148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:47.235148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:47.235148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:47.235148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:47.235148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:47.235148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:47.236156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:47.236156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:47.236156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:47.236156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:47.236156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:47.236156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:47.236156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:47.236156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:47.236156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:47.236156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:47.236156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:53:47.236156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:47.238637 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 119/239] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:47.244175 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b93d5c-c9c4-4808-99ce-160d467a27eb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:47.251094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e01cbf-166a-433f-9e07-89aa3a6bf461 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:47.356839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b93d5c-c9c4-4808-99ce-160d467a27eb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:47.372185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:47.412248 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e01cbf-166a-433f-9e07-89aa3a6bf461 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:47.434405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917437d-7bd2-4e10-a2da-40dff72dd8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:47.435494 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917437d-7bd2-4e10-a2da-40dff72dd8bd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:47.436014 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917437d-7bd2-4e10-a2da-40dff72dd8bd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:47.436520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4917437d-7bd2-4e10-a2da-40dff72dd8bd None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:47.437486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917437d-7bd2-4e10-a2da-40dff72dd8bd None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:47.438068 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917437d-7bd2-4e10-a2da-40dff72dd8bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:47.439681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4917437d-7bd2-4e10-a2da-40dff72dd8bd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:47.461720 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 121/240] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:47 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:47.467180 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917437d-7bd2-4e10-a2da-40dff72dd8bd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:47.491845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e235884-fc93-46d9-9d93-51a2d7bd45d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:47.491845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e235884-fc93-46d9-9d93-51a2d7bd45d2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:47.491845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e235884-fc93-46d9-9d93-51a2d7bd45d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:47.491845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e235884-fc93-46d9-9d93-51a2d7bd45d2 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:47.492236 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e235884-fc93-46d9-9d93-51a2d7bd45d2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:47.494485 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e235884-fc93-46d9-9d93-51a2d7bd45d2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:47.500686 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 120/241] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:53:47 2022] GET /identity/v3/domains/default => generated 199 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:47.501507 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e235884-fc93-46d9-9d93-51a2d7bd45d2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:47.517138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:47.612548 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e235884-fc93-46d9-9d93-51a2d7bd45d2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:47.657266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c566f00-fb13-4f61-ae5f-e9d6350e455c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:47.658547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c566f00-fb13-4f61-ae5f-e9d6350e455c None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:47.660613 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4917437d-7bd2-4e10-a2da-40dff72dd8bd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:47.661106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c566f00-fb13-4f61-ae5f-e9d6350e455c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:47.663005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c566f00-fb13-4f61-ae5f-e9d6350e455c None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:47.663972 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c566f00-fb13-4f61-ae5f-e9d6350e455c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:47.665947 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c566f00-fb13-4f61-ae5f-e9d6350e455c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:47.688201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:47.704613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 121/242] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:53:47 2022] POST /identity/v3/users => generated 293 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:53:47.724947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35413828-23ac-4941-a4ff-8992ea655a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:47.726267 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35413828-23ac-4941-a4ff-8992ea655a47 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:47.726757 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35413828-23ac-4941-a4ff-8992ea655a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:47.727233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35413828-23ac-4941-a4ff-8992ea655a47 None admin] PATH_INFO: `/v3/users/76e237d326f94330bf09fc3166abc758` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:47.743966 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35413828-23ac-4941-a4ff-8992ea655a47 None admin] RBAC: Authorizing `identity:get_user(user_id=76e237d326f94330bf09fc3166abc758)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:47.745813 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35413828-23ac-4941-a4ff-8992ea655a47 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:47.747861 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 122/243] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:47 2022] GET /identity/v3/users/76e237d326f94330bf09fc3166abc758 => generated 297 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:47.753971 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c566f00-fb13-4f61-ae5f-e9d6350e455c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:47.822400 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf31e02-8503-46ea-bc10-62836e0d6eaa None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:47.822761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf31e02-8503-46ea-bc10-62836e0d6eaa None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:47.822967 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf31e02-8503-46ea-bc10-62836e0d6eaa None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:47.824390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 123/244] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:47.829625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87247963-f70a-4f04-958d-cc2c5c31d862 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:47.829919 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87247963-f70a-4f04-958d-cc2c5c31d862 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:47.830204 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87247963-f70a-4f04-958d-cc2c5c31d862 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:47.850948 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-87247963-f70a-4f04-958d-cc2c5c31d862 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:47.852812 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-87247963-f70a-4f04-958d-cc2c5c31d862 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:47.910217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c566f00-fb13-4f61-ae5f-e9d6350e455c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:47.949374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 124/245] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:53:47.964033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec768356-d45a-48e9-b206-167e2356c215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:47.964714 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec768356-d45a-48e9-b206-167e2356c215 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:47.965003 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec768356-d45a-48e9-b206-167e2356c215 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:47.965287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec768356-d45a-48e9-b206-167e2356c215 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:47.997499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87247963-f70a-4f04-958d-cc2c5c31d862 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:48.000862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:48.041039 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:48.048863 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec768356-d45a-48e9-b206-167e2356c215 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:48.050857 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec768356-d45a-48e9-b206-167e2356c215 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:48.134475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87247963-f70a-4f04-958d-cc2c5c31d862 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:48.179103 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 122/246] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:53:48.184724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec768356-d45a-48e9-b206-167e2356c215 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:48.184924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058342d7-572c-4365-bd5d-6e9573f6e5d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:48.185719 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058342d7-572c-4365-bd5d-6e9573f6e5d6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:48.186015 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058342d7-572c-4365-bd5d-6e9573f6e5d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:48.186297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058342d7-572c-4365-bd5d-6e9573f6e5d6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:48.207127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-058342d7-572c-4365-bd5d-6e9573f6e5d6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:48.221258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-058342d7-572c-4365-bd5d-6e9573f6e5d6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:48.221258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:48.221258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:48.221258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:48.221258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:48.221258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:48.221258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:48.221258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:48.221258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:48.221258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:48.221258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:48.221258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:48.221258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:48.221258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:48.221258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:48.221258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:48.222218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:48.222218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:48.222218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:48.222218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:48.222218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:48.222218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:48.222218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:48.222218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:48.222218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:48.222218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:48.222218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:48.222218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:53:48.222218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:48.222218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:48.222218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:48.222218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:48.223007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:48.223007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:48.223007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:48.223007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:48.223007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:48.223007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:48.223007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:48.223007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:48.223007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:48.223007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:48.223007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:53:48.223007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:53:48.223007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 125/247] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:53:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:53:48.228891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058342d7-572c-4365-bd5d-6e9573f6e5d6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:48.310441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:48.326091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:48.335263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec768356-d45a-48e9-b206-167e2356c215 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:48.362773 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-058342d7-572c-4365-bd5d-6e9573f6e5d6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:48.375998 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93297ead-ac3d-464e-a9cd-582510ae251b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:48.376825 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93297ead-ac3d-464e-a9cd-582510ae251b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:48.377117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93297ead-ac3d-464e-a9cd-582510ae251b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:48.377393 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93297ead-ac3d-464e-a9cd-582510ae251b None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:48.378682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93297ead-ac3d-464e-a9cd-582510ae251b None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:48.380188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93297ead-ac3d-464e-a9cd-582510ae251b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:48.382387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 123/248] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:48 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:48.389764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93297ead-ac3d-464e-a9cd-582510ae251b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:48.410534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffab2a-9fc5-42eb-9bee-76eecbf56887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:48.411274 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffab2a-9fc5-42eb-9bee-76eecbf56887 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:48.411564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffab2a-9fc5-42eb-9bee-76eecbf56887 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:48.411859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ffab2a-9fc5-42eb-9bee-76eecbf56887 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:48.412629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ffab2a-9fc5-42eb-9bee-76eecbf56887 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:48.412993 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ffab2a-9fc5-42eb-9bee-76eecbf56887 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:48.414510 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ffab2a-9fc5-42eb-9bee-76eecbf56887 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:48.424744 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 126/249] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:48.432023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffab2a-9fc5-42eb-9bee-76eecbf56887 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:48.525899 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93297ead-ac3d-464e-a9cd-582510ae251b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:48.553720 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ffab2a-9fc5-42eb-9bee-76eecbf56887 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:48.579781 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baeec54-1cd6-4033-bf91-795a4cb41b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:48.580894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baeec54-1cd6-4033-bf91-795a4cb41b12 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:48.581382 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baeec54-1cd6-4033-bf91-795a4cb41b12 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:48.582176 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2baeec54-1cd6-4033-bf91-795a4cb41b12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:48.583285 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baeec54-1cd6-4033-bf91-795a4cb41b12 None admin] RBAC: Adding query filter params (role.id=b7283d3771c94f458ebfcaef5784afa0, scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7, user.id=76e237d326f94330bf09fc3166abc758) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:48.584073 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baeec54-1cd6-4033-bf91-795a4cb41b12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:48.586903 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2baeec54-1cd6-4033-bf91-795a4cb41b12 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:48.587546 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:48.596798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 124/250] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:48 2022] GET /identity/v3/role_assignments?user.id=76e237d326f94330bf09fc3166abc758&scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7&role.id=b7283d3771c94f458ebfcaef5784afa0 => generated 259 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:48.599700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c54086-80b7-4b05-ac02-52f7c5661cd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:48.600723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c54086-80b7-4b05-ac02-52f7c5661cd3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:48.601075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c54086-80b7-4b05-ac02-52f7c5661cd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:48.601424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c54086-80b7-4b05-ac02-52f7c5661cd3 None admin] PATH_INFO: `/v3/users/6cda4100d68a4b28aeaef2a83ad10cce` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:48.603316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c54086-80b7-4b05-ac02-52f7c5661cd3 None admin] RBAC: Authorizing `identity:get_user(user_id=6cda4100d68a4b28aeaef2a83ad10cce)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:48.604930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c54086-80b7-4b05-ac02-52f7c5661cd3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:48.606895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 127/251] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:48 2022] GET /identity/v3/users/6cda4100d68a4b28aeaef2a83ad10cce => generated 283 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:48.612983 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baeec54-1cd6-4033-bf91-795a4cb41b12 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:48.647112 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:48.746918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2baeec54-1cd6-4033-bf91-795a4cb41b12 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:48.795404 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4913d8-1f3e-4b59-a4cb-a2fea6e99f26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:48.796104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4913d8-1f3e-4b59-a4cb-a2fea6e99f26 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:48.796392 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4913d8-1f3e-4b59-a4cb-a2fea6e99f26 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:48.796670 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4913d8-1f3e-4b59-a4cb-a2fea6e99f26 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:48.881040 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4913d8-1f3e-4b59-a4cb-a2fea6e99f26 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:48.882537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4913d8-1f3e-4b59-a4cb-a2fea6e99f26 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:48.910277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:48.914178 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19720fd-1a3a-4ef1-b824-52e608cddbf6 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:48.914474 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19720fd-1a3a-4ef1-b824-52e608cddbf6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:48.914756 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f19720fd-1a3a-4ef1-b824-52e608cddbf6 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:48.916415 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 128/252] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:48.937476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f418a1-cfcc-489f-9876-54007c8ee6ec None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:48.937841 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f418a1-cfcc-489f-9876-54007c8ee6ec None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:48.938109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f418a1-cfcc-489f-9876-54007c8ee6ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:48.958072 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-05f418a1-cfcc-489f-9876-54007c8ee6ec None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:48.959256 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-05f418a1-cfcc-489f-9876-54007c8ee6ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:49.023154 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 125/253] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:53:49.024920 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca11c9f-628d-4084-b3e8-cab6166bf160 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:49.025407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca11c9f-628d-4084-b3e8-cab6166bf160 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:49.025944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca11c9f-628d-4084-b3e8-cab6166bf160 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:49.027424 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 126/254] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:49.032494 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca11c9f-628d-4084-b3e8-cab6166bf160 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:49.033231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 129/255] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:49.034740 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94257e3-5a86-4bbb-8e61-67ae17230827 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:49.034964 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94257e3-5a86-4bbb-8e61-67ae17230827 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:49.035230 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94257e3-5a86-4bbb-8e61-67ae17230827 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:49.052431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b94257e3-5a86-4bbb-8e61-67ae17230827 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:49.053422 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b94257e3-5a86-4bbb-8e61-67ae17230827 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:49.156254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca11c9f-628d-4084-b3e8-cab6166bf160 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:49.166106 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 130/256] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:49.166986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94257e3-5a86-4bbb-8e61-67ae17230827 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:49.204925 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:49.212322 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d48a12-2192-457c-bcd7-ab3c5f04f47c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:49.213597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d48a12-2192-457c-bcd7-ab3c5f04f47c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:49.214070 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d48a12-2192-457c-bcd7-ab3c5f04f47c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:49.214525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d48a12-2192-457c-bcd7-ab3c5f04f47c None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:49.217794 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d48a12-2192-457c-bcd7-ab3c5f04f47c None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:49.219554 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d48a12-2192-457c-bcd7-ab3c5f04f47c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:49.221668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 127/257] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:49 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:49.227493 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d48a12-2192-457c-bcd7-ab3c5f04f47c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:49.333952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d48a12-2192-457c-bcd7-ab3c5f04f47c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:49.368771 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:49.371543 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80072887-953b-4f9d-a3af-597e304de80b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:49.372661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80072887-953b-4f9d-a3af-597e304de80b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:49.373126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80072887-953b-4f9d-a3af-597e304de80b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:49.373559 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80072887-953b-4f9d-a3af-597e304de80b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:49.375121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80072887-953b-4f9d-a3af-597e304de80b None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7, user.id=6cda4100d68a4b28aeaef2a83ad10cce) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:49.375533 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80072887-953b-4f9d-a3af-597e304de80b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:49.377659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80072887-953b-4f9d-a3af-597e304de80b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:49.387380 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 128/258] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:49 2022] GET /identity/v3/role_assignments?user.id=6cda4100d68a4b28aeaef2a83ad10cce&scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:49.388616 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80072887-953b-4f9d-a3af-597e304de80b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:49.439002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94257e3-5a86-4bbb-8e61-67ae17230827 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:49.515427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900157a3-c1d2-46a2-bbff-ea4d9bec941f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:49.517327 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900157a3-c1d2-46a2-bbff-ea4d9bec941f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:49.517840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:49.519849 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900157a3-c1d2-46a2-bbff-ea4d9bec941f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:49.520359 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-900157a3-c1d2-46a2-bbff-ea4d9bec941f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:49.540033 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-900157a3-c1d2-46a2-bbff-ea4d9bec941f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:49.558279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-900157a3-c1d2-46a2-bbff-ea4d9bec941f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:49.558279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:49.558279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:49.558279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:49.558279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:49.558279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:49.558279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:49.558279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:49.558279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:49.558279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:49.558279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:49.558279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:49.558279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:49.558279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:49.558279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:49.558279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:49.559240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:49.559240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:49.559240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:49.559240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:49.559240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:49.559240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:49.559240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:49.559240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:49.559240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:49.559240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:49.559240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:49.559240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:53:49.559240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:49.559240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:49.559240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:49.559240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:49.560086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:49.560086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:49.560086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:49.560086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:49.560086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:49.560086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:49.560086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:49.560086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:49.560086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:49.560086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:49.560086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:49.560086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:53:49.560086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 131/259] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 393 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:49.567218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900157a3-c1d2-46a2-bbff-ea4d9bec941f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:49.596710 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80072887-953b-4f9d-a3af-597e304de80b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:49.633652 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec2ab3-3981-4b2b-b354-be68c622dd26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:49.634748 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec2ab3-3981-4b2b-b354-be68c622dd26 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:49.635178 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec2ab3-3981-4b2b-b354-be68c622dd26 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:49.635621 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ec2ab3-3981-4b2b-b354-be68c622dd26 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:49.647420 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66ec2ab3-3981-4b2b-b354-be68c622dd26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:49.656188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-66ec2ab3-3981-4b2b-b354-be68c622dd26 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:53:49.656188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:49.656188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:49.656188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:49.656188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:49.656188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:49.656188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:49.656188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:49.656188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:49.656188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:49.656188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:49.656188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:49.656188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:49.656188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:49.656188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:49.656188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:49.657277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:49.657277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:49.657277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:49.657277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:49.657277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:49.657277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:49.657277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:49.657277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:49.657277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:49.657277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:49.657277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:49.657277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:49.657277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:49.657277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:49.657277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:49.657277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:49.658072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:49.658072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:49.658072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:49.658072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:49.658072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:49.658072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:49.658072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:49.658072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:49.658072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:49.658072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:49.658072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:53:49.658072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:49.660608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 129/260] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:53:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:49.666778 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec2ab3-3981-4b2b-b354-be68c622dd26 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:49.786575 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ec2ab3-3981-4b2b-b354-be68c622dd26 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:49.809883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-900157a3-c1d2-46a2-bbff-ea4d9bec941f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:49.810618 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:49.828789 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7dee8-6414-41fe-ab58-054b32648d86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:49.831392 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7dee8-6414-41fe-ab58-054b32648d86 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:49.833300 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7dee8-6414-41fe-ab58-054b32648d86 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:49.833779 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:49.833924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf7dee8-6414-41fe-ab58-054b32648d86 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:49.834863 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7dee8-6414-41fe-ab58-054b32648d86 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:49.836680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7dee8-6414-41fe-ab58-054b32648d86 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:49.838330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf7dee8-6414-41fe-ab58-054b32648d86 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:49.859045 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 130/261] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:53:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:49.865155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7dee8-6414-41fe-ab58-054b32648d86 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:49.884576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645887af-a868-47e2-96c0-73f8d60c287f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:49.885951 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645887af-a868-47e2-96c0-73f8d60c287f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:49.886515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645887af-a868-47e2-96c0-73f8d60c287f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:49.886942 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-645887af-a868-47e2-96c0-73f8d60c287f None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:49.887934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645887af-a868-47e2-96c0-73f8d60c287f None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:49.888438 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645887af-a868-47e2-96c0-73f8d60c287f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:49.890587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-645887af-a868-47e2-96c0-73f8d60c287f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:49.900696 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 132/262] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:49 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:49.906467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645887af-a868-47e2-96c0-73f8d60c287f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:50.018488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-645887af-a868-47e2-96c0-73f8d60c287f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:50.056470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84fc71-d1d1-4f72-b087-e08694a99c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:50.057707 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84fc71-d1d1-4f72-b087-e08694a99c1a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:50.058196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84fc71-d1d1-4f72-b087-e08694a99c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:50.058647 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a84fc71-d1d1-4f72-b087-e08694a99c1a None admin] PATH_INFO: `/v3/users/fafa43aa9cb04a5aa9b5259fa1f93e17` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:50.060114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84fc71-d1d1-4f72-b087-e08694a99c1a None admin] RBAC: Authorizing `identity:get_user(user_id=fafa43aa9cb04a5aa9b5259fa1f93e17)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:50.062388 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a84fc71-d1d1-4f72-b087-e08694a99c1a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:50.064325 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 133/263] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:49 2022] GET /identity/v3/users/fafa43aa9cb04a5aa9b5259fa1f93e17 => generated 297 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:50.070324 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84fc71-d1d1-4f72-b087-e08694a99c1a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:50.129905 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:50.143574 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf7dee8-6414-41fe-ab58-054b32648d86 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:50.194909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a84fc71-d1d1-4f72-b087-e08694a99c1a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:50.237856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2399c2d-63a1-4881-9759-86080f49e0d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:50.238764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2399c2d-63a1-4881-9759-86080f49e0d4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:50.239137 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2399c2d-63a1-4881-9759-86080f49e0d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:50.239426 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2399c2d-63a1-4881-9759-86080f49e0d4 None admin] PATH_INFO: `/v3/users/6cda4100d68a4b28aeaef2a83ad10cce` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:50.244561 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2399c2d-63a1-4881-9759-86080f49e0d4 None admin] RBAC: Authorizing `identity:get_user(user_id=6cda4100d68a4b28aeaef2a83ad10cce)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:50.246663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2399c2d-63a1-4881-9759-86080f49e0d4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:50.250749 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fed42c-efac-4c79-83c9-f1b3a2d4e79c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:50.251919 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fed42c-efac-4c79-83c9-f1b3a2d4e79c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:50.252386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fed42c-efac-4c79-83c9-f1b3a2d4e79c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:50.252835 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fed42c-efac-4c79-83c9-f1b3a2d4e79c None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:50.253554 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 131/264] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:49 2022] GET /identity/v3/users/6cda4100d68a4b28aeaef2a83ad10cce => generated 283 bytes in 385 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:50.254839 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2399c2d-63a1-4881-9759-86080f49e0d4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:50.255625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fed42c-efac-4c79-83c9-f1b3a2d4e79c None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:50.258822 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9fed42c-efac-4c79-83c9-f1b3a2d4e79c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:50.263138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 134/265] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:50 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:50.268715 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fed42c-efac-4c79-83c9-f1b3a2d4e79c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:50.360820 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:50.428992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fed42c-efac-4c79-83c9-f1b3a2d4e79c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:50.444307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:50.474574 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2399c2d-63a1-4881-9759-86080f49e0d4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:50.476757 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7bc76-3a24-402d-8e81-9d1027d28d99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:50.477682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7bc76-3a24-402d-8e81-9d1027d28d99 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:50.477970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7bc76-3a24-402d-8e81-9d1027d28d99 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:50.478253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc7bc76-3a24-402d-8e81-9d1027d28d99 None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7/users/fafa43aa9cb04a5aa9b5259fa1f93e17/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:50.484677 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7bc76-3a24-402d-8e81-9d1027d28d99 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8be03eb22fb4d4d9fb156c025f259d7, user_id=fafa43aa9cb04a5aa9b5259fa1f93e17, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:50.486395 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc7bc76-3a24-402d-8e81-9d1027d28d99 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:50.523482 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 135/266] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:53:50 2022] PUT /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7/users/fafa43aa9cb04a5aa9b5259fa1f93e17/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:53:50.531768 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1e580b2-390e-42c6-b03a-c91a69c837ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:50.532522 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e580b2-390e-42c6-b03a-c91a69c837ef None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:50.532814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e580b2-390e-42c6-b03a-c91a69c837ef None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:50.533099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1e580b2-390e-42c6-b03a-c91a69c837ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:50.613540 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e580b2-390e-42c6-b03a-c91a69c837ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:50.614965 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1e580b2-390e-42c6-b03a-c91a69c837ef None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:50.666910 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:50.727571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 132/267] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:50 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 473 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:53:50.737299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7bc76-3a24-402d-8e81-9d1027d28d99 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:50.875599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc7bc76-3a24-402d-8e81-9d1027d28d99 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:50.918365 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a0f13-d106-4ffa-a8e2-318ad51814a1 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:50.918701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a0f13-d106-4ffa-a8e2-318ad51814a1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:50.918934 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4a0f13-d106-4ffa-a8e2-318ad51814a1 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:50.920443 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 133/268] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:50.925866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f220d-6894-4da2-b3c6-c06b452cefed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:50.926372 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c113531-874e-4a78-9886-5393df318947 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:50.926673 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c113531-874e-4a78-9886-5393df318947 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:50.926770 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f220d-6894-4da2-b3c6-c06b452cefed None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:50.926963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c113531-874e-4a78-9886-5393df318947 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:50.927067 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f220d-6894-4da2-b3c6-c06b452cefed None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:50.927309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-288f220d-6894-4da2-b3c6-c06b452cefed None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:50.929827 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f220d-6894-4da2-b3c6-c06b452cefed None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:50.931456 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-288f220d-6894-4da2-b3c6-c06b452cefed None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:50.945269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 136/269] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:50 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:50.946304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f220d-6894-4da2-b3c6-c06b452cefed None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:50.962157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-5c113531-874e-4a78-9886-5393df318947 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:50.963393 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5c113531-874e-4a78-9886-5393df318947 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:51.017796 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:51.051169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 134/270] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:53:51.078790 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c113531-874e-4a78-9886-5393df318947 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:51.079745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-288f220d-6894-4da2-b3c6-c06b452cefed None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:51.132226 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d91217c0-261c-4a1c-93b1-a5ff42f2f1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:51.132813 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91217c0-261c-4a1c-93b1-a5ff42f2f1d2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:51.133107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91217c0-261c-4a1c-93b1-a5ff42f2f1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:51.133400 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d91217c0-261c-4a1c-93b1-a5ff42f2f1d2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:51.134241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91217c0-261c-4a1c-93b1-a5ff42f2f1d2 None admin] RBAC: Adding query filter params (role.id=1cc9b5d798c648c986537d006320581f, scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7, user.id=6cda4100d68a4b28aeaef2a83ad10cce) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:51.134796 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91217c0-261c-4a1c-93b1-a5ff42f2f1d2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:51.137465 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d91217c0-261c-4a1c-93b1-a5ff42f2f1d2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:51.147026 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 137/271] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:50 2022] GET /identity/v3/role_assignments?user.id=6cda4100d68a4b28aeaef2a83ad10cce&scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7&role.id=1cc9b5d798c648c986537d006320581f => generated 608 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:51.158533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46f1fc-21b2-457e-b5fa-ae8c330f1dbe None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:51.159146 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46f1fc-21b2-457e-b5fa-ae8c330f1dbe None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:51.159576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe46f1fc-21b2-457e-b5fa-ae8c330f1dbe None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:51.161216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 138/272] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:53:51.167255 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15ee9f-0dd6-4698-8c5f-2e58cf8cfabe None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:51.167771 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15ee9f-0dd6-4698-8c5f-2e58cf8cfabe None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:51.168258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db15ee9f-0dd6-4698-8c5f-2e58cf8cfabe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:51.201314 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-db15ee9f-0dd6-4698-8c5f-2e58cf8cfabe None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:51.202543 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-db15ee9f-0dd6-4698-8c5f-2e58cf8cfabe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:51.233586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:51.276978 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c113531-874e-4a78-9886-5393df318947 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:51.352189 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 139/273] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:53:51.361127 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9e234-fa09-4934-8954-e55868b93632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:51.361997 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9e234-fa09-4934-8954-e55868b93632 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:51.362292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9e234-fa09-4934-8954-e55868b93632 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:51.362576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b9e234-fa09-4934-8954-e55868b93632 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:51.363807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9e234-fa09-4934-8954-e55868b93632 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:51.365330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b9e234-fa09-4934-8954-e55868b93632 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:51.367028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 135/274] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:53:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 289 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:51.372707 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15ee9f-0dd6-4698-8c5f-2e58cf8cfabe None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:51.378465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9e234-fa09-4934-8954-e55868b93632 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:51.414287 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:51.505857 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db15ee9f-0dd6-4698-8c5f-2e58cf8cfabe None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:51.509796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b9e234-fa09-4934-8954-e55868b93632 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:51.545873 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781a7cc4-c204-4ec6-9c3e-48bed82a2e2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:51.546140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9320a7e-e4dc-4b49-81df-5163406cb5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:51.547024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a7cc4-c204-4ec6-9c3e-48bed82a2e2c None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:51.547301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9320a7e-e4dc-4b49-81df-5163406cb5ee None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:51.547603 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a7cc4-c204-4ec6-9c3e-48bed82a2e2c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:51.547751 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9320a7e-e4dc-4b49-81df-5163406cb5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:51.548148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9320a7e-e4dc-4b49-81df-5163406cb5ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:51.548321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781a7cc4-c204-4ec6-9c3e-48bed82a2e2c None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:51.549179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a7cc4-c204-4ec6-9c3e-48bed82a2e2c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:51.550786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781a7cc4-c204-4ec6-9c3e-48bed82a2e2c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:51.559614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9320a7e-e4dc-4b49-81df-5163406cb5ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:51.568759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-c9320a7e-e4dc-4b49-81df-5163406cb5ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:51.568759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:51.568759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:51.568759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:51.568759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:51.568759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:51.568759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:51.568759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:51.568759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:51.568759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:51.568759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:51.568759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:51.568759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:51.568759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:51.568759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:51.568759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:51.569893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:51.569893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:51.569893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:51.569893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:51.569893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:51.569893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:51.569893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:51.569893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:51.569893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:51.569893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:51.569893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:51.569893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:51.569893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:51.569893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:51.569893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:51.569893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:51.570921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:51.570921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:51.570921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:51.570921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:51.570921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:51.570921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:51.570921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:51.570921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:51.570921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:51.570921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:51.570921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:51.570921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:51.573642 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 136/275] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:51.582403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9320a7e-e4dc-4b49-81df-5163406cb5ee None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:51.584875 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 140/276] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:53:51 2022] POST /identity/v3/users => generated 293 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:53:51.601508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:51.617227 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2422eb-5c7f-4a8a-9e2a-ea6384e4a266 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:51.617557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2422eb-5c7f-4a8a-9e2a-ea6384e4a266 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:51.617802 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c2422eb-5c7f-4a8a-9e2a-ea6384e4a266 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:51.624856 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 141/277] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:53:51.627631 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26308cc6-4be9-4aa9-a55e-e9881fde7e70 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:51.628044 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26308cc6-4be9-4aa9-a55e-e9881fde7e70 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:51.628401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26308cc6-4be9-4aa9-a55e-e9881fde7e70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:51.652703 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-26308cc6-4be9-4aa9-a55e-e9881fde7e70 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:51.654031 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-26308cc6-4be9-4aa9-a55e-e9881fde7e70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:51.736424 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9320a7e-e4dc-4b49-81df-5163406cb5ee None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:51.740064 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 142/278] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:51.763172 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26308cc6-4be9-4aa9-a55e-e9881fde7e70 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:51.778698 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0672d6-64ac-4478-ac4e-982becb9dbd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:51.779774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0672d6-64ac-4478-ac4e-982becb9dbd4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:51.780352 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0672d6-64ac-4478-ac4e-982becb9dbd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:51.780828 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb0672d6-64ac-4478-ac4e-982becb9dbd4 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:51.782018 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0672d6-64ac-4478-ac4e-982becb9dbd4 None admin] RBAC: Adding query filter params (name=member) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:51.782542 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0672d6-64ac-4478-ac4e-982becb9dbd4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:51.784240 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb0672d6-64ac-4478-ac4e-982becb9dbd4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:51.787490 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:51.794223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 137/279] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:51.799923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0672d6-64ac-4478-ac4e-982becb9dbd4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:51.896985 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:51.901242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26308cc6-4be9-4aa9-a55e-e9881fde7e70 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:51.942140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb0672d6-64ac-4478-ac4e-982becb9dbd4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:51.943071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3b8cc-c264-4617-a17d-399d2eafcc8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:51.944080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3b8cc-c264-4617-a17d-399d2eafcc8a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:51.944372 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3b8cc-c264-4617-a17d-399d2eafcc8a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:51.944654 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bb3b8cc-c264-4617-a17d-399d2eafcc8a None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:51.959584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bb3b8cc-c264-4617-a17d-399d2eafcc8a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:51.968094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-5bb3b8cc-c264-4617-a17d-399d2eafcc8a None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:53:51.968094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:51.968094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:51.968094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:51.968094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:51.968094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:51.968094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:51.968094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:51.968094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:51.968094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:51.968094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:51.968094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:51.968094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:51.968094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:51.968094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:51.968094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:51.969101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:51.969101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:51.969101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:51.969101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:51.969101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:51.969101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:51.969101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:51.969101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:51.969101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:51.969101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:51.969101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:51.969101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:53:51.969101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:51.969101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:51.969101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:51.969101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:51.969894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:51.969894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:51.969894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:51.969894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:51.969894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:51.969894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:51.969894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:51.969894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:51.969894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:51.969894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:51.969894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:53:51.969894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:53:51.969894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 143/280] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:51 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:53:51.975306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3b8cc-c264-4617-a17d-399d2eafcc8a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:51.992361 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7846830-8b00-4671-b529-fa18496af217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:51.993253 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7846830-8b00-4671-b529-fa18496af217 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:51.994319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7846830-8b00-4671-b529-fa18496af217 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:51.994319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7846830-8b00-4671-b529-fa18496af217 None admin] PATH_INFO: `/v3/users/28bb38908d8b4772bc87811c65c4c460` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:52.016272 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7846830-8b00-4671-b529-fa18496af217 None admin] RBAC: Authorizing `identity:get_user(user_id=28bb38908d8b4772bc87811c65c4c460)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:52.021630 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7846830-8b00-4671-b529-fa18496af217 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:52.025631 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 138/281] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:51 2022] GET /identity/v3/users/28bb38908d8b4772bc87811c65c4c460 => generated 293 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:52.030839 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7846830-8b00-4671-b529-fa18496af217 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:52.076522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:52.099135 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bb3b8cc-c264-4617-a17d-399d2eafcc8a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:52.145635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efdd08-8467-4e14-920c-6a4399e52c92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:52.146672 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efdd08-8467-4e14-920c-6a4399e52c92 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:52.147237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efdd08-8467-4e14-920c-6a4399e52c92 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:52.147741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23efdd08-8467-4e14-920c-6a4399e52c92 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:52.148798 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23efdd08-8467-4e14-920c-6a4399e52c92 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:52.149941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23efdd08-8467-4e14-920c-6a4399e52c92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:52.151658 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23efdd08-8467-4e14-920c-6a4399e52c92 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:52.159952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7846830-8b00-4671-b529-fa18496af217 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:52.162568 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 144/282] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:51 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:52.168414 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efdd08-8467-4e14-920c-6a4399e52c92 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:52.192237 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:52.202775 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04584e1e-c78f-4fda-a4f5-cc3b0685a03e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:52.203515 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04584e1e-c78f-4fda-a4f5-cc3b0685a03e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:52.203815 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04584e1e-c78f-4fda-a4f5-cc3b0685a03e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:52.204126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04584e1e-c78f-4fda-a4f5-cc3b0685a03e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:52.205947 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04584e1e-c78f-4fda-a4f5-cc3b0685a03e None admin] RBAC: Adding query filter params (role.id=9c81542451e54c0ea22a20c026be6be7, user.id=28bb38908d8b4772bc87811c65c4c460, scope.system=all) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:52.205947 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04584e1e-c78f-4fda-a4f5-cc3b0685a03e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:52.210241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04584e1e-c78f-4fda-a4f5-cc3b0685a03e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:52.215852 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 139/283] 10.222.0.85 () {56 vars in 1308 bytes} [Wed May 25 15:53:52 2022] GET /identity/v3/role_assignments?user.id=28bb38908d8b4772bc87811c65c4c460&scope.system=all&role.id=9c81542451e54c0ea22a20c026be6be7 => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:52.344027 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23efdd08-8467-4e14-920c-6a4399e52c92 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:52.399533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3d56b-3242-405c-ba00-dd3e46213352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:52.400499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3d56b-3242-405c-ba00-dd3e46213352 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:52.400784 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3d56b-3242-405c-ba00-dd3e46213352 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:52.401069 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff3d56b-3242-405c-ba00-dd3e46213352 None admin] PATH_INFO: `/v3/users/76e237d326f94330bf09fc3166abc758` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:52.402328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3d56b-3242-405c-ba00-dd3e46213352 None admin] RBAC: Authorizing `identity:get_user(user_id=76e237d326f94330bf09fc3166abc758)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:52.403861 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff3d56b-3242-405c-ba00-dd3e46213352 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:52.405758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 145/284] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:52 2022] GET /identity/v3/users/76e237d326f94330bf09fc3166abc758 => generated 297 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:52.411764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04584e1e-c78f-4fda-a4f5-cc3b0685a03e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:52.538025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04584e1e-c78f-4fda-a4f5-cc3b0685a03e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:52.578887 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fafcfc1-57f7-4d72-8c61-be0b7ab4d3cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:52.579702 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafcfc1-57f7-4d72-8c61-be0b7ab4d3cd None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:52.580022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafcfc1-57f7-4d72-8c61-be0b7ab4d3cd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:52.580301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fafcfc1-57f7-4d72-8c61-be0b7ab4d3cd None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:52.581489 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafcfc1-57f7-4d72-8c61-be0b7ab4d3cd None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:52.582951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fafcfc1-57f7-4d72-8c61-be0b7ab4d3cd None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:52.584694 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 140/285] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:52 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:52.590993 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3d56b-3242-405c-ba00-dd3e46213352 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:52.605991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:52.734583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff3d56b-3242-405c-ba00-dd3e46213352 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:52.772587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86cebcbd-f664-4f42-b286-c4d474236271 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:52.773848 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cebcbd-f664-4f42-b286-c4d474236271 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:52.774334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cebcbd-f664-4f42-b286-c4d474236271 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:52.774853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86cebcbd-f664-4f42-b286-c4d474236271 None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7/users/76e237d326f94330bf09fc3166abc758/roles/b7283d3771c94f458ebfcaef5784afa0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:52.779895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cebcbd-f664-4f42-b286-c4d474236271 None admin] RBAC: Authorizing `identity:create_grant(project_id=a8be03eb22fb4d4d9fb156c025f259d7, user_id=76e237d326f94330bf09fc3166abc758, role_id=b7283d3771c94f458ebfcaef5784afa0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:52.781739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86cebcbd-f664-4f42-b286-c4d474236271 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:52.812909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 146/286] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:53:52 2022] PUT /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7/users/76e237d326f94330bf09fc3166abc758/roles/b7283d3771c94f458ebfcaef5784afa0 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:53:53.315505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322c8cc-a4ec-4411-8280-205b13f9cb13 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:53.315793 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322c8cc-a4ec-4411-8280-205b13f9cb13 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:53.316119 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0322c8cc-a4ec-4411-8280-205b13f9cb13 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:53.317503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 141/287] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:53:53.323973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b81c77-8870-4a47-a388-ac0aa8245337 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:53.324562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b81c77-8870-4a47-a388-ac0aa8245337 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:53.325011 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93b81c77-8870-4a47-a388-ac0aa8245337 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:53.352999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-93b81c77-8870-4a47-a388-ac0aa8245337 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:53.354201 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-93b81c77-8870-4a47-a388-ac0aa8245337 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:53.444490 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:53.454755 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 147/288] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:53.492495 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322c8cc-a4ec-4411-8280-205b13f9cb13 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:53.568115 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:53.620695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0322c8cc-a4ec-4411-8280-205b13f9cb13 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:53.665889 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94d162-edff-4623-934c-4ee367a3b637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:53.667073 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94d162-edff-4623-934c-4ee367a3b637 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:53.667586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94d162-edff-4623-934c-4ee367a3b637 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:53.668091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94d162-edff-4623-934c-4ee367a3b637 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:53.684673 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94d162-edff-4623-934c-4ee367a3b637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:53.692612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-6c94d162-edff-4623-934c-4ee367a3b637 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:53.692612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:53.692612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:53.692612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:53.692612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:53.692612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:53.692612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:53.692612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:53.692612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:53.692612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:53.692612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:53.692612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:53.692612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:53.692612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:53.692612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:53.692612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:53.693534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:53.693534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:53.693534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:53.693534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:53.693534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:53.693534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:53.693534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:53.693534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:53.693534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:53.693534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:53.693534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:53.693534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:53.693534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:53.693534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:53.693534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:53.693534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:53.694335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:53.694335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:53.694335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:53.694335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:53.694335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:53.694335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:53.694335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:53.694335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:53.694335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:53.694335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:53.694335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:53.694335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:53.694942 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 142/289] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:53 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:53.699537 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b81c77-8870-4a47-a388-ac0aa8245337 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:53.832171 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93b81c77-8870-4a47-a388-ac0aa8245337 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:53.881426 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857040fe-9d6d-439a-9f08-a466ae9adf8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:53.882120 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857040fe-9d6d-439a-9f08-a466ae9adf8d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:53.882415 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857040fe-9d6d-439a-9f08-a466ae9adf8d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:53.882702 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857040fe-9d6d-439a-9f08-a466ae9adf8d None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:53.883467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857040fe-9d6d-439a-9f08-a466ae9adf8d None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:53.883787 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857040fe-9d6d-439a-9f08-a466ae9adf8d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:53.885217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857040fe-9d6d-439a-9f08-a466ae9adf8d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:53.899749 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 148/290] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:53 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:53.904794 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94d162-edff-4623-934c-4ee367a3b637 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:54.023429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94d162-edff-4623-934c-4ee367a3b637 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:54.044151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:54.065786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ffabe6-3628-42e9-ae14-1a2563bdfcb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:54.066691 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ffabe6-3628-42e9-ae14-1a2563bdfcb1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:54.066980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ffabe6-3628-42e9-ae14-1a2563bdfcb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:54.067261 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2ffabe6-3628-42e9-ae14-1a2563bdfcb1 None admin] PATH_INFO: `/v3/users/fafa43aa9cb04a5aa9b5259fa1f93e17` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:54.072545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ffabe6-3628-42e9-ae14-1a2563bdfcb1 None admin] RBAC: Authorizing `identity:get_user(user_id=fafa43aa9cb04a5aa9b5259fa1f93e17)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:54.074036 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2ffabe6-3628-42e9-ae14-1a2563bdfcb1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:54.075890 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 143/291] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:53 2022] GET /identity/v3/users/fafa43aa9cb04a5aa9b5259fa1f93e17 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:54.081157 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857040fe-9d6d-439a-9f08-a466ae9adf8d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:54.160360 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:54.207919 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857040fe-9d6d-439a-9f08-a466ae9adf8d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:54.257516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c04795d-9e7b-4491-9a35-b7280f6febdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:54.258200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04795d-9e7b-4491-9a35-b7280f6febdf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:54.258499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04795d-9e7b-4491-9a35-b7280f6febdf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:54.258786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c04795d-9e7b-4491-9a35-b7280f6febdf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:54.375995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04795d-9e7b-4491-9a35-b7280f6febdf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:54.377489 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c04795d-9e7b-4491-9a35-b7280f6febdf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:54.464078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:54.517462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 149/292] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:53:54.522672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ffabe6-3628-42e9-ae14-1a2563bdfcb1 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:54.625217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6bff8-41f9-4c47-9d8b-49cf71be4f0b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:54.625546 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6bff8-41f9-4c47-9d8b-49cf71be4f0b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:54.625793 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a6bff8-41f9-4c47-9d8b-49cf71be4f0b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:54.627185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 150/293] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:54.637110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5c1df-962f-41f8-a713-090a6c627a53 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:54.637677 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5c1df-962f-41f8-a713-090a6c627a53 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:54.638155 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5c1df-962f-41f8-a713-090a6c627a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:54.667850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2ffabe6-3628-42e9-ae14-1a2563bdfcb1 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:54.680850 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-94f5c1df-962f-41f8-a713-090a6c627a53 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:54.682368 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-94f5c1df-962f-41f8-a713-090a6c627a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:54.717094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872ff39-6d0e-41f3-affe-ec55c20df72d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:54.718324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872ff39-6d0e-41f3-affe-ec55c20df72d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:54.718839 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872ff39-6d0e-41f3-affe-ec55c20df72d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:54.719332 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0872ff39-6d0e-41f3-affe-ec55c20df72d None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:54.721570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872ff39-6d0e-41f3-affe-ec55c20df72d None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:54.724145 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0872ff39-6d0e-41f3-affe-ec55c20df72d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:54.727270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 144/294] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:54 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:54.733068 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872ff39-6d0e-41f3-affe-ec55c20df72d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:54.811289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 151/295] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:54.833523 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:54.846653 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5c1df-962f-41f8-a713-090a6c627a53 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:54.920885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0872ff39-6d0e-41f3-affe-ec55c20df72d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:54.977834 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c276e7-7e4c-4cfb-b7e0-5d54cc6842ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:54.978576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c276e7-7e4c-4cfb-b7e0-5d54cc6842ff None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:54.978937 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c276e7-7e4c-4cfb-b7e0-5d54cc6842ff None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:54.979287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c276e7-7e4c-4cfb-b7e0-5d54cc6842ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:54.980242 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c276e7-7e4c-4cfb-b7e0-5d54cc6842ff None admin] RBAC: Adding query filter params (role.id=9c81542451e54c0ea22a20c026be6be7, scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7, user.id=fafa43aa9cb04a5aa9b5259fa1f93e17) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:54.980631 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c276e7-7e4c-4cfb-b7e0-5d54cc6842ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:54.982213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c276e7-7e4c-4cfb-b7e0-5d54cc6842ff None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:54.988764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 145/296] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:54 2022] GET /identity/v3/role_assignments?user.id=fafa43aa9cb04a5aa9b5259fa1f93e17&scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7&role.id=9c81542451e54c0ea22a20c026be6be7 => generated 608 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:54.989937 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a796b1-d2dd-47e4-89a0-2f6ef3cca0b5 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:54.990303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a796b1-d2dd-47e4-89a0-2f6ef3cca0b5 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:54.990656 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80a796b1-d2dd-47e4-89a0-2f6ef3cca0b5 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:54.991970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 146/297] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:54.993306 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af802b-f034-41e5-a820-9c1079f229cf None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:54.993809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af802b-f034-41e5-a820-9c1079f229cf None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:54.994289 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83af802b-f034-41e5-a820-9c1079f229cf None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:54.996124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 147/298] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:54.998146 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060db5fe-8852-4dd7-8d52-88be98e09ee3 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:54.998913 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060db5fe-8852-4dd7-8d52-88be98e09ee3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:54.999894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060db5fe-8852-4dd7-8d52-88be98e09ee3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:55.007186 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:55.017071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5c1df-962f-41f8-a713-090a6c627a53 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:55.023581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-060db5fe-8852-4dd7-8d52-88be98e09ee3 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:55.025265 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-060db5fe-8852-4dd7-8d52-88be98e09ee3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:55.062345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343590f-d3f4-4341-a8d0-281eb836c735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:55.063678 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343590f-d3f4-4341-a8d0-281eb836c735 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:55.064477 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343590f-d3f4-4341-a8d0-281eb836c735 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:55.064764 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343590f-d3f4-4341-a8d0-281eb836c735 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:55.081284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343590f-d3f4-4341-a8d0-281eb836c735 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:55.092328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-1343590f-d3f4-4341-a8d0-281eb836c735 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:53:55.092328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:55.092328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:55.092328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:55.092328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:55.092328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:55.092328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:55.092328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:55.092328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:55.092328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:55.092328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:55.092328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:55.092328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:55.092328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:55.092328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:55.092328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:55.093337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:55.093337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:55.093337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:55.093337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:55.093337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:55.093337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:55.093337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:55.093337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:55.093337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:55.093337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:55.093337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:55.093337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:53:55.093337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:55.093337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:55.093337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:55.093337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:55.095164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:55.095164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:55.095164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:55.095164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:55.095164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:55.095164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:55.095164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:55.095164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:55.095164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:55.095164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:55.095164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:53:55.095164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:53:55.095164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 152/299] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:55.096176 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c66c8-87a3-4012-af62-df573a8639d5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:55.096176 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c66c8-87a3-4012-af62-df573a8639d5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:55.096176 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c8c66c8-87a3-4012-af62-df573a8639d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:55.115468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3c8c66c8-87a3-4012-af62-df573a8639d5 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:55.116558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 148/300] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:55.118157 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3c8c66c8-87a3-4012-af62-df573a8639d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:55.119143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060db5fe-8852-4dd7-8d52-88be98e09ee3 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:55.198283 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 153/301] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:55.198789 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:55.199155 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c66c8-87a3-4012-af62-df573a8639d5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:55.257761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060db5fe-8852-4dd7-8d52-88be98e09ee3 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:55.302455 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:55.302672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070542c0-7543-4d64-b4d2-db03c1fb79fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:55.303427 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070542c0-7543-4d64-b4d2-db03c1fb79fb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:55.303722 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070542c0-7543-4d64-b4d2-db03c1fb79fb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:55.304086 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070542c0-7543-4d64-b4d2-db03c1fb79fb None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:55.305795 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070542c0-7543-4d64-b4d2-db03c1fb79fb None admin] RBAC: Adding query filter params (name=reader) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:55.305795 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070542c0-7543-4d64-b4d2-db03c1fb79fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:55.306555 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070542c0-7543-4d64-b4d2-db03c1fb79fb None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:55.318442 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 149/302] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:55 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:55.319853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070542c0-7543-4d64-b4d2-db03c1fb79fb None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:55.330582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c8c66c8-87a3-4012-af62-df573a8639d5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:55.374336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162296a-16a6-4348-abbf-61bb805de3d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:55.375186 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162296a-16a6-4348-abbf-61bb805de3d4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:55.375478 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162296a-16a6-4348-abbf-61bb805de3d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:55.375758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9162296a-16a6-4348-abbf-61bb805de3d4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:55.377030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162296a-16a6-4348-abbf-61bb805de3d4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:55.378501 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9162296a-16a6-4348-abbf-61bb805de3d4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:55.380185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 154/303] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:53:55 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:55.380960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162296a-16a6-4348-abbf-61bb805de3d4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:55.442167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070542c0-7543-4d64-b4d2-db03c1fb79fb None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:55.455318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:55.476747 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53eb15-7322-436d-91f4-6bbebcefc8f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:55.477562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53eb15-7322-436d-91f4-6bbebcefc8f0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:55.477853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53eb15-7322-436d-91f4-6bbebcefc8f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:55.478136 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53eb15-7322-436d-91f4-6bbebcefc8f0 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:55.483463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9162296a-16a6-4348-abbf-61bb805de3d4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:55.488536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53eb15-7322-436d-91f4-6bbebcefc8f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:55.496355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-cb53eb15-7322-436d-91f4-6bbebcefc8f0 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:55.496355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:55.496355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:55.496355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:55.496355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:55.496355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:55.496355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:55.496355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:55.496355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:55.496355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:55.496355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:55.496355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:55.496355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:55.496355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:55.496355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:55.496355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:55.497303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:55.497303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:55.497303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:55.497303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:55.497303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:55.497303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:55.497303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:55.497303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:55.497303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:55.497303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:55.497303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:55.497303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:55.497303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:55.497303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:55.497303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:55.497303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:55.498087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:55.498087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:55.498087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:55.498087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:55.498087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:55.498087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:55.498087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:55.498087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:55.498087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:55.498087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:55.498087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:55.498087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:55.498087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 150/304] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:55.498750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53eb15-7322-436d-91f4-6bbebcefc8f0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:55.518322 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-089bc289-1b2b-4b04-88df-e5668c6e911e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:55.519154 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bc289-1b2b-4b04-88df-e5668c6e911e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:55.519442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bc289-1b2b-4b04-88df-e5668c6e911e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:55.519726 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-089bc289-1b2b-4b04-88df-e5668c6e911e None admin] PATH_INFO: `/v3/users/1ab68bdb0bce44c6b7a7c995da8f87eb` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:55.532949 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089bc289-1b2b-4b04-88df-e5668c6e911e None admin] RBAC: Authorizing `identity:get_user(user_id=1ab68bdb0bce44c6b7a7c995da8f87eb)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:55.534468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-089bc289-1b2b-4b04-88df-e5668c6e911e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:55.536180 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 155/305] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:55 2022] GET /identity/v3/users/1ab68bdb0bce44c6b7a7c995da8f87eb => generated 293 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:55.537470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8690b-25ec-4b32-b329-e52537d05167 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:55.537769 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8690b-25ec-4b32-b329-e52537d05167 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:55.538046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d8690b-25ec-4b32-b329-e52537d05167 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:55.539277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 156/306] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:55.540005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8690b-25ec-4b32-b329-e52537d05167 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:55.550662 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:55.600254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53eb15-7322-436d-91f4-6bbebcefc8f0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:55.632396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74150bf6-1305-4405-9198-d57a0079baff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:55.633072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74150bf6-1305-4405-9198-d57a0079baff None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:55.633361 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74150bf6-1305-4405-9198-d57a0079baff None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:55.633650 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74150bf6-1305-4405-9198-d57a0079baff None admin] PATH_INFO: `/v3/groups` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:55.634343 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74150bf6-1305-4405-9198-d57a0079baff None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:55.635797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74150bf6-1305-4405-9198-d57a0079baff None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:55.641093 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60d8690b-25ec-4b32-b329-e52537d05167 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:55.651122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 151/307] 10.222.0.85 () {60 vars in 1134 bytes} [Wed May 25 15:53:55 2022] POST /identity/v3/groups => generated 229 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:53:55.651952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74150bf6-1305-4405-9198-d57a0079baff None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:55.680612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8adfd2-6996-46d2-b62c-a5fe8d649973 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:55.681313 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8adfd2-6996-46d2-b62c-a5fe8d649973 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:55.681604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8adfd2-6996-46d2-b62c-a5fe8d649973 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:55.681888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8adfd2-6996-46d2-b62c-a5fe8d649973 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:55.682673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8adfd2-6996-46d2-b62c-a5fe8d649973 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:55.683009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8adfd2-6996-46d2-b62c-a5fe8d649973 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:55.684512 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8adfd2-6996-46d2-b62c-a5fe8d649973 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:55.687187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:55.695364 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 157/308] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:55.696978 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa1f80-0b63-4e04-ac77-0ab4981077c5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:55.697377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa1f80-0b63-4e04-ac77-0ab4981077c5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:55.697761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daa1f80-0b63-4e04-ac77-0ab4981077c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:55.717767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6daa1f80-0b63-4e04-ac77-0ab4981077c5 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:55.719074 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6daa1f80-0b63-4e04-ac77-0ab4981077c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:55.771498 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74150bf6-1305-4405-9198-d57a0079baff None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:55.789230 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:55.795417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 158/309] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:55.796737 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa1f80-0b63-4e04-ac77-0ab4981077c5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:55.818878 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa3d65-a61e-4460-9c23-5262d7610a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:55.820118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa3d65-a61e-4460-9c23-5262d7610a4b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:55.820588 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa3d65-a61e-4460-9c23-5262d7610a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:55.821078 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fa3d65-a61e-4460-9c23-5262d7610a4b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:55.822138 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa3d65-a61e-4460-9c23-5262d7610a4b None admin] RBAC: Adding query filter params (role.id=b7283d3771c94f458ebfcaef5784afa0, user.id=1ab68bdb0bce44c6b7a7c995da8f87eb, scope.system=all) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:55.822633 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa3d65-a61e-4460-9c23-5262d7610a4b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:55.824375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fa3d65-a61e-4460-9c23-5262d7610a4b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:55.830486 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 152/310] 10.222.0.85 () {56 vars in 1308 bytes} [Wed May 25 15:53:55 2022] GET /identity/v3/role_assignments?user.id=1ab68bdb0bce44c6b7a7c995da8f87eb&scope.system=all&role.id=b7283d3771c94f458ebfcaef5784afa0 => generated 226 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:55.831342 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa3d65-a61e-4460-9c23-5262d7610a4b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:55.937599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daa1f80-0b63-4e04-ac77-0ab4981077c5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:55.958075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:55.965845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fa3d65-a61e-4460-9c23-5262d7610a4b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:55.989867 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23552a5f-3dff-4868-b1d3-001331ddcb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:55.991317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23552a5f-3dff-4868-b1d3-001331ddcb82 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:55.991800 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23552a5f-3dff-4868-b1d3-001331ddcb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:55.992668 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23552a5f-3dff-4868-b1d3-001331ddcb82 None admin] PATH_INFO: `/v3/users/28bb38908d8b4772bc87811c65c4c460` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:55.994565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23552a5f-3dff-4868-b1d3-001331ddcb82 None admin] RBAC: Authorizing `identity:get_user(user_id=28bb38908d8b4772bc87811c65c4c460)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:55.996980 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23552a5f-3dff-4868-b1d3-001331ddcb82 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:56.000133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 159/311] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:55 2022] GET /identity/v3/users/28bb38908d8b4772bc87811c65c4c460 => generated 293 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:56.003109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546a02e-402b-445d-9351-424aeceeb0c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:56.007990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546a02e-402b-445d-9351-424aeceeb0c6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:56.007990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546a02e-402b-445d-9351-424aeceeb0c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:56.007990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2546a02e-402b-445d-9351-424aeceeb0c6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:56.008260 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23552a5f-3dff-4868-b1d3-001331ddcb82 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:56.020501 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2546a02e-402b-445d-9351-424aeceeb0c6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:56.030042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-2546a02e-402b-445d-9351-424aeceeb0c6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:53:56.030042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:56.030042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:56.030042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:56.030042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:56.030042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:56.030042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:56.030042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:56.030042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:56.030042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:56.030042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:56.030042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:56.030042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:56.030042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:56.030042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:56.030042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:56.031213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:56.031213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:56.031213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:56.031213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:56.031213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:56.031213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:56.031213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:56.031213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:56.031213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:56.031213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:56.031213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:56.031213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:56.031213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:56.031213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:56.031213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:56.031213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:56.032257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:56.032257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:56.032257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:56.032257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:56.032257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:56.032257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:56.032257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:56.032257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:56.032257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:56.032257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:56.032257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:53:56.032257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:56.034912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 153/312] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:55 2022] GET /identity/v3/roles/reader => generated 84 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:53:56.041353 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546a02e-402b-445d-9351-424aeceeb0c6 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:56.094076 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:56.135991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23552a5f-3dff-4868-b1d3-001331ddcb82 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:56.155333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2546a02e-402b-445d-9351-424aeceeb0c6 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:56.177751 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7409d8b-adc7-4b5a-b18f-92e3cb5bfa9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:56.178903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7409d8b-adc7-4b5a-b18f-92e3cb5bfa9d None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:56.179190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7409d8b-adc7-4b5a-b18f-92e3cb5bfa9d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:56.179466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7409d8b-adc7-4b5a-b18f-92e3cb5bfa9d None admin] PATH_INFO: `/v3/system/users/28bb38908d8b4772bc87811c65c4c460/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:56.181138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7409d8b-adc7-4b5a-b18f-92e3cb5bfa9d None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=28bb38908d8b4772bc87811c65c4c460, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:56.183092 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7409d8b-adc7-4b5a-b18f-92e3cb5bfa9d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:56.194879 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35919c86-85f0-46a8-9ea6-611658121e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:56.195952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35919c86-85f0-46a8-9ea6-611658121e00 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:56.196240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 160/313] 10.222.0.85 () {58 vars in 1334 bytes} [Wed May 25 15:53:56 2022] PUT /identity/v3/system/users/28bb38908d8b4772bc87811c65c4c460/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:53:56.196818 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35919c86-85f0-46a8-9ea6-611658121e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:56.197389 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35919c86-85f0-46a8-9ea6-611658121e00 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:56.198882 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35919c86-85f0-46a8-9ea6-611658121e00 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:56.199459 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35919c86-85f0-46a8-9ea6-611658121e00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:56.201314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35919c86-85f0-46a8-9ea6-611658121e00 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:56.218823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 154/314] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:56.224786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7409d8b-adc7-4b5a-b18f-92e3cb5bfa9d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:56.360761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7409d8b-adc7-4b5a-b18f-92e3cb5bfa9d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:56.397229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851a41c-95ad-4600-a639-df86aca79843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:56.398929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851a41c-95ad-4600-a639-df86aca79843 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:56.399287 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851a41c-95ad-4600-a639-df86aca79843 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:56.399633 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9851a41c-95ad-4600-a639-df86aca79843 None admin] PATH_INFO: `/v3/users/76e237d326f94330bf09fc3166abc758` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:56.400576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:56.402044 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851a41c-95ad-4600-a639-df86aca79843 None admin] RBAC: Authorizing `identity:get_user(user_id=76e237d326f94330bf09fc3166abc758)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:56.403903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9851a41c-95ad-4600-a639-df86aca79843 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:56.406159 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 161/315] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:56 2022] GET /identity/v3/users/76e237d326f94330bf09fc3166abc758 => generated 297 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:56.411923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35919c86-85f0-46a8-9ea6-611658121e00 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:56.433514 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:56.517113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35919c86-85f0-46a8-9ea6-611658121e00 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:56.551421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adb75c-c4ee-487a-8032-b1e9449bd624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:56.552116 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adb75c-c4ee-487a-8032-b1e9449bd624 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:56.552412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adb75c-c4ee-487a-8032-b1e9449bd624 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:56.552697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08adb75c-c4ee-487a-8032-b1e9449bd624 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:56.621682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08adb75c-c4ee-487a-8032-b1e9449bd624 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:56.623168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08adb75c-c4ee-487a-8032-b1e9449bd624 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:56.644166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:56.722519 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 155/316] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:53:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 15:53:56.727773 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851a41c-95ad-4600-a639-df86aca79843 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:56.835544 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9851a41c-95ad-4600-a639-df86aca79843 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:56.870492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1eba090-6977-4ce1-a248-b8b592b69a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:56.871766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eba090-6977-4ce1-a248-b8b592b69a47 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:56.872374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eba090-6977-4ce1-a248-b8b592b69a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:56.872844 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1eba090-6977-4ce1-a248-b8b592b69a47 None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:56.874221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eba090-6977-4ce1-a248-b8b592b69a47 None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:56.875966 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1eba090-6977-4ce1-a248-b8b592b69a47 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:56.878182 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 162/317] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:53:56 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:56.883659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adb75c-c4ee-487a-8032-b1e9449bd624 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:56.993206 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08adb75c-c4ee-487a-8032-b1e9449bd624 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:57.010849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:57.029575 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848c9db-d297-4771-b937-6bfbea865e3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:57.030614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848c9db-d297-4771-b937-6bfbea865e3b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:57.031172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848c9db-d297-4771-b937-6bfbea865e3b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:57.031647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848c9db-d297-4771-b937-6bfbea865e3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:57.032700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848c9db-d297-4771-b937-6bfbea865e3b None admin] RBAC: Adding query filter params (role.id=b7283d3771c94f458ebfcaef5784afa0, scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7, user.id=76e237d326f94330bf09fc3166abc758) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:57.033265 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848c9db-d297-4771-b937-6bfbea865e3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:57.034958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1848c9db-d297-4771-b937-6bfbea865e3b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:57.042264 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 156/318] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:53:56 2022] GET /identity/v3/role_assignments?user.id=76e237d326f94330bf09fc3166abc758&scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7&role.id=b7283d3771c94f458ebfcaef5784afa0 => generated 608 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:58.026381 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88339ebd-2bf1-4001-a89e-9a2faaed8f36 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:58.026937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88339ebd-2bf1-4001-a89e-9a2faaed8f36 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:58.026937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88339ebd-2bf1-4001-a89e-9a2faaed8f36 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:58.028471 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 163/319] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:53:58.033988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0e034-e9c5-4057-9993-3387a25e716b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:58.034404 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0e034-e9c5-4057-9993-3387a25e716b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:58.034571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ca0e034-e9c5-4057-9993-3387a25e716b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:58.053152 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-0ca0e034-e9c5-4057-9993-3387a25e716b None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:58.054360 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0ca0e034-e9c5-4057-9993-3387a25e716b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:58.087358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3f63d-3b4e-412e-807e-01f59ea1c92f None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:58.087666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3f63d-3b4e-412e-807e-01f59ea1c92f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:58.088020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e3f63d-3b4e-412e-807e-01f59ea1c92f None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:58.089614 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 164/320] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:53:58.095274 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255c7b7-b235-4bee-abdc-dcaff6321a9a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:58.095634 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255c7b7-b235-4bee-abdc-dcaff6321a9a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:58.096001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5255c7b7-b235-4bee-abdc-dcaff6321a9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:58.113618 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5255c7b7-b235-4bee-abdc-dcaff6321a9a None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:58.115040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5255c7b7-b235-4bee-abdc-dcaff6321a9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:58.125267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 157/321] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:58.147852 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0e034-e9c5-4057-9993-3387a25e716b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:58.188157 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:58.196044 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 165/322] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:53:58.216775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255c7b7-b235-4bee-abdc-dcaff6321a9a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:58.256932 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ca0e034-e9c5-4057-9993-3387a25e716b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:58.270765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:58.295815 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee032f-06b1-43b0-a5f2-b64076b2ac8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:58.297035 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee032f-06b1-43b0-a5f2-b64076b2ac8e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:58.297394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee032f-06b1-43b0-a5f2-b64076b2ac8e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:58.297680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ee032f-06b1-43b0-a5f2-b64076b2ac8e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:58.301481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee032f-06b1-43b0-a5f2-b64076b2ac8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:58.301481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ee032f-06b1-43b0-a5f2-b64076b2ac8e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:58.306704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 158/323] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:53:58 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:58.308149 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee032f-06b1-43b0-a5f2-b64076b2ac8e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:58.326972 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5255c7b7-b235-4bee-abdc-dcaff6321a9a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:58.364649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567e37f-77c9-4fc9-9cd4-61fef05ed10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:58.365892 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567e37f-77c9-4fc9-9cd4-61fef05ed10f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:58.366353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567e37f-77c9-4fc9-9cd4-61fef05ed10f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:58.366838 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b567e37f-77c9-4fc9-9cd4-61fef05ed10f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:58.379961 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b567e37f-77c9-4fc9-9cd4-61fef05ed10f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:58.388349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b567e37f-77c9-4fc9-9cd4-61fef05ed10f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:53:58.388349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:58.388349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:58.388349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:58.388349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:58.388349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:58.388349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:58.388349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:58.388349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:58.388349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:58.388349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:58.388349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:58.388349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:58.388349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:58.388349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:58.388349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:58.389378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:58.389378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:58.389378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:58.389378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:58.389378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:58.389378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:58.389378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:58.389378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:58.389378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:58.389378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:58.389378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:58.389378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:53:58.389378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:58.389378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:58.389378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:58.389378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:58.390262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:58.390262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:58.390262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:58.390262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:58.390262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:58.390262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:58.390262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:58.390262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:58.390262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:58.390262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:58.390262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:53:58.390262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:53:58.390262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 166/324] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:58 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:53:58.395888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567e37f-77c9-4fc9-9cd4-61fef05ed10f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:58.414922 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ee032f-06b1-43b0-a5f2-b64076b2ac8e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:58.449736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742cbae0-9c34-42cc-bf2c-caf3eb0b1a9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:58.450410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cbae0-9c34-42cc-bf2c-caf3eb0b1a9c None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:58.450703 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cbae0-9c34-42cc-bf2c-caf3eb0b1a9c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:58.450984 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742cbae0-9c34-42cc-bf2c-caf3eb0b1a9c None admin] PATH_INFO: `/v3/groups` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:58.451682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cbae0-9c34-42cc-bf2c-caf3eb0b1a9c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:58.453264 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-742cbae0-9c34-42cc-bf2c-caf3eb0b1a9c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:58.458774 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:58.469477 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 159/325] 10.222.0.85 () {60 vars in 1134 bytes} [Wed May 25 15:53:58 2022] POST /identity/v3/groups => generated 226 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:53:58.514892 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b567e37f-77c9-4fc9-9cd4-61fef05ed10f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:58.555189 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec4259-15f5-4316-a2e8-068e9305f424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:58.555802 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec4259-15f5-4316-a2e8-068e9305f424 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:58.556123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec4259-15f5-4316-a2e8-068e9305f424 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:58.556412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ec4259-15f5-4316-a2e8-068e9305f424 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:58.557194 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec4259-15f5-4316-a2e8-068e9305f424 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:58.557515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec4259-15f5-4316-a2e8-068e9305f424 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:58.558975 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ec4259-15f5-4316-a2e8-068e9305f424 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:58.562371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65643d6c-c9bc-4e73-805d-87f3f1de7301 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:58.562678 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65643d6c-c9bc-4e73-805d-87f3f1de7301 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:58.562965 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65643d6c-c9bc-4e73-805d-87f3f1de7301 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:58.564559 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 160/326] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:53:58.569586 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 167/327] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:58.570319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76d498-7e3b-4a4b-ba76-d758ec97fb2a None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:58.570742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76d498-7e3b-4a4b-ba76-d758ec97fb2a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:58.571041 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d76d498-7e3b-4a4b-ba76-d758ec97fb2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:58.575155 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec4259-15f5-4316-a2e8-068e9305f424 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:58.593121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8d76d498-7e3b-4a4b-ba76-d758ec97fb2a None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:53:58.594424 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8d76d498-7e3b-4a4b-ba76-d758ec97fb2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:53:58.632304 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:58.673459 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 161/328] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:53:58.692198 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ec4259-15f5-4316-a2e8-068e9305f424 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:58.696829 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76d498-7e3b-4a4b-ba76-d758ec97fb2a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:58.714961 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:58.736356 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33093ec-b367-4477-ab28-b504d4ffd670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:58.737920 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33093ec-b367-4477-ab28-b504d4ffd670 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:58.738355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33093ec-b367-4477-ab28-b504d4ffd670 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:58.738803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33093ec-b367-4477-ab28-b504d4ffd670 None admin] PATH_INFO: `/v3/users/1ab68bdb0bce44c6b7a7c995da8f87eb` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:58.740084 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33093ec-b367-4477-ab28-b504d4ffd670 None admin] RBAC: Authorizing `identity:get_user(user_id=1ab68bdb0bce44c6b7a7c995da8f87eb)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:58.742554 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33093ec-b367-4477-ab28-b504d4ffd670 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:58.744277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 168/329] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:58 2022] GET /identity/v3/users/1ab68bdb0bce44c6b7a7c995da8f87eb => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:53:58.750391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33093ec-b367-4477-ab28-b504d4ffd670 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:58.827548 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d76d498-7e3b-4a4b-ba76-d758ec97fb2a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:58.873821 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4ce45-5e4d-4702-998e-16dce0103b91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:58.874481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4ce45-5e4d-4702-998e-16dce0103b91 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:58.875194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4ce45-5e4d-4702-998e-16dce0103b91 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:58.875494 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e4ce45-5e4d-4702-998e-16dce0103b91 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:58.877829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33093ec-b367-4477-ab28-b504d4ffd670 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:58.888315 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e4ce45-5e4d-4702-998e-16dce0103b91 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:53:58.897810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-39e4ce45-5e4d-4702-998e-16dce0103b91 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:58.897810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:53:58.897810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:53:58.897810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:53:58.897810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:53:58.897810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:53:58.897810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:53:58.897810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:53:58.897810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:53:58.897810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:53:58.897810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:53:58.897810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:53:58.897810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:53:58.897810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:53:58.897810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:53:58.897810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:53:58.898729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:53:58.898729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:53:58.898729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:53:58.898729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:53:58.898729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:53:58.898729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:53:58.898729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:53:58.898729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:53:58.898729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:53:58.898729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:53:58.898729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:53:58.898729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:53:58.898729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:53:58.898729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:53:58.898729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:53:58.898729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:53:58.899525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:53:58.899525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:53:58.899525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:53:58.899525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:53:58.899525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:53:58.899525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:53:58.899525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:53:58.899525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:53:58.899525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:53:58.899525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:53:58.899525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:53:58.899525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:53:58.899525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 162/330] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:53:58 2022] GET /identity/v3/roles/member => generated 84 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:53:58.906505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4ce45-5e4d-4702-998e-16dce0103b91 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:58.923334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec03c2-e6e6-4b18-bfc4-3db3d1773c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:58.924359 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec03c2-e6e6-4b18-bfc4-3db3d1773c07 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:58.924717 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec03c2-e6e6-4b18-bfc4-3db3d1773c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:58.925066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ec03c2-e6e6-4b18-bfc4-3db3d1773c07 None admin] PATH_INFO: `/v3/system/users/1ab68bdb0bce44c6b7a7c995da8f87eb/roles/b7283d3771c94f458ebfcaef5784afa0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:58.926704 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec03c2-e6e6-4b18-bfc4-3db3d1773c07 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=1ab68bdb0bce44c6b7a7c995da8f87eb, role_id=b7283d3771c94f458ebfcaef5784afa0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:58.928158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ec03c2-e6e6-4b18-bfc4-3db3d1773c07 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:58.939960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 169/331] 10.222.0.85 () {58 vars in 1334 bytes} [Wed May 25 15:53:58 2022] PUT /identity/v3/system/users/1ab68bdb0bce44c6b7a7c995da8f87eb/roles/b7283d3771c94f458ebfcaef5784afa0 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:53:58.947763 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:59.033386 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e4ce45-5e4d-4702-998e-16dce0103b91 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:59.073877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6f2d9-e1a6-4254-b59e-a5f4fb99e720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:59.074629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6f2d9-e1a6-4254-b59e-a5f4fb99e720 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:59.074926 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6f2d9-e1a6-4254-b59e-a5f4fb99e720 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:59.075206 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ba6f2d9-e1a6-4254-b59e-a5f4fb99e720 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:59.076016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6f2d9-e1a6-4254-b59e-a5f4fb99e720 None admin] RBAC: Adding query filter params (name=member) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:59.076335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6f2d9-e1a6-4254-b59e-a5f4fb99e720 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:59.077975 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ba6f2d9-e1a6-4254-b59e-a5f4fb99e720 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:59.088005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 163/332] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:53:58 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:59.097016 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec03c2-e6e6-4b18-bfc4-3db3d1773c07 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:59.166206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:59.223196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ec03c2-e6e6-4b18-bfc4-3db3d1773c07 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:59.259021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c636388-e6b9-4243-8fdb-95cd3b765315 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:59.260351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c636388-e6b9-4243-8fdb-95cd3b765315 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:59.260854 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c636388-e6b9-4243-8fdb-95cd3b765315 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:59.261320 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c636388-e6b9-4243-8fdb-95cd3b765315 None admin] PATH_INFO: `/v3/users/28bb38908d8b4772bc87811c65c4c460` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:59.263303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c636388-e6b9-4243-8fdb-95cd3b765315 None admin] RBAC: Authorizing `identity:get_user(user_id=28bb38908d8b4772bc87811c65c4c460)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:59.264814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c636388-e6b9-4243-8fdb-95cd3b765315 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:59.266747 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 170/333] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:53:59 2022] GET /identity/v3/users/28bb38908d8b4772bc87811c65c4c460 => generated 293 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:53:59.272940 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6f2d9-e1a6-4254-b59e-a5f4fb99e720 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:53:59.383074 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:53:59.391745 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ba6f2d9-e1a6-4254-b59e-a5f4fb99e720 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:53:59.431391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e179e120-0932-468f-a213-85bd0963b19c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:53:59.432097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179e120-0932-468f-a213-85bd0963b19c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:53:59.432386 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179e120-0932-468f-a213-85bd0963b19c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:53:59.432666 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e179e120-0932-468f-a213-85bd0963b19c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:53:59.433513 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179e120-0932-468f-a213-85bd0963b19c None admin] RBAC: Adding query filter params (role.id=9c81542451e54c0ea22a20c026be6be7, user.id=28bb38908d8b4772bc87811c65c4c460, scope.system=all) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:53:59.433832 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179e120-0932-468f-a213-85bd0963b19c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:53:59.435297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e179e120-0932-468f-a213-85bd0963b19c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:53:59.452824 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 164/334] 10.222.0.85 () {56 vars in 1308 bytes} [Wed May 25 15:53:59 2022] GET /identity/v3/role_assignments?user.id=28bb38908d8b4772bc87811c65c4c460&scope.system=all&role.id=9c81542451e54c0ea22a20c026be6be7 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:01.082727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e89f62-690a-4a8b-9ae4-f13abc4ef000 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:01.083298 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e89f62-690a-4a8b-9ae4-f13abc4ef000 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:01.083298 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e89f62-690a-4a8b-9ae4-f13abc4ef000 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:01.085321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 171/335] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:01.090771 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ec8c4-7597-4096-9f40-0947ae9e14f6 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:01.091214 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ec8c4-7597-4096-9f40-0947ae9e14f6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:01.091415 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ec8c4-7597-4096-9f40-0947ae9e14f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:01.116845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-0d6ec8c4-7597-4096-9f40-0947ae9e14f6 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:01.118063 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0d6ec8c4-7597-4096-9f40-0947ae9e14f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:01.201881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 165/336] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:01.242264 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e89f62-690a-4a8b-9ae4-f13abc4ef000 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:01.393493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e89f62-690a-4a8b-9ae4-f13abc4ef000 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:01.418621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:01.435079 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df2f138c-5799-4e07-a15a-7a7a07f604af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:01.436036 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f138c-5799-4e07-a15a-7a7a07f604af None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:01.436410 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f138c-5799-4e07-a15a-7a7a07f604af None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:01.436698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df2f138c-5799-4e07-a15a-7a7a07f604af None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:01.453057 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df2f138c-5799-4e07-a15a-7a7a07f604af None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:01.461551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-df2f138c-5799-4e07-a15a-7a7a07f604af None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:54:01.461551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:01.461551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:01.461551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:01.461551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:01.461551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:01.461551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:01.461551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:01.461551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:01.461551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:01.461551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:01.461551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:01.461551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:01.461551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:01.461551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:01.461551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:01.462548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:01.462548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:01.462548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:01.462548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:01.462548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:01.462548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:01.462548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:01.462548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:01.462548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:01.462548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:01.462548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:01.462548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:01.462548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:01.462548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:01.462548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:01.462548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:01.463383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:01.463383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:01.463383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:01.463383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:01.463383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:01.463383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:01.463383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:01.463383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:01.463383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:01.463383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:01.463383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:54:01.463383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:01.463383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 172/337] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:01.469379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ec8c4-7597-4096-9f40-0947ae9e14f6 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:01.602069 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:01.602913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bb954-ad8b-46b0-9ad7-046ffdd12f72 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:01.603425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bb954-ad8b-46b0-9ad7-046ffdd12f72 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:01.603941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907bb954-ad8b-46b0-9ad7-046ffdd12f72 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:01.607102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 173/338] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:01 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:01.617760 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7995c38-3132-48e5-9be5-e8b3cd3187e6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:01.618549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7995c38-3132-48e5-9be5-e8b3cd3187e6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:01.619023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7995c38-3132-48e5-9be5-e8b3cd3187e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:01.637167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ec8c4-7597-4096-9f40-0947ae9e14f6 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:01.651404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e7995c38-3132-48e5-9be5-e8b3cd3187e6 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:01.652910 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e7995c38-3132-48e5-9be5-e8b3cd3187e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:01.718251 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb599cd-b7d0-499c-b229-d9b3f147c086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:01.719011 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb599cd-b7d0-499c-b229-d9b3f147c086 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:01.719312 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb599cd-b7d0-499c-b229-d9b3f147c086 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:01.719596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb599cd-b7d0-499c-b229-d9b3f147c086 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:01.720395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb599cd-b7d0-499c-b229-d9b3f147c086 None admin] RBAC: Adding query filter params (name=member) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:01.720710 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb599cd-b7d0-499c-b229-d9b3f147c086 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:01.722121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb599cd-b7d0-499c-b229-d9b3f147c086 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:01.738400 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 166/339] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:54:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:01.744120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb599cd-b7d0-499c-b229-d9b3f147c086 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:01.773729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 174/340] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:01.813434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7995c38-3132-48e5-9be5-e8b3cd3187e6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:01.907655 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb599cd-b7d0-499c-b229-d9b3f147c086 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:01.975746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf306768-fa65-417e-b292-c77b342171fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:01.976453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf306768-fa65-417e-b292-c77b342171fb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:01.976746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf306768-fa65-417e-b292-c77b342171fb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:01.977031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf306768-fa65-417e-b292-c77b342171fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:01.996964 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:02.022146 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7995c38-3132-48e5-9be5-e8b3cd3187e6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:02.047240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:02.101547 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841e1f1-248e-4469-8cbe-01af2093cb44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:02.103218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841e1f1-248e-4469-8cbe-01af2093cb44 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:02.103903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841e1f1-248e-4469-8cbe-01af2093cb44 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:02.104380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3841e1f1-248e-4469-8cbe-01af2093cb44 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:02.125732 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3841e1f1-248e-4469-8cbe-01af2093cb44 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:02.140583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3841e1f1-248e-4469-8cbe-01af2093cb44 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:54:02.140583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:02.140583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:02.140583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:02.140583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:02.140583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:02.140583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:02.140583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:02.140583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:02.140583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:02.140583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:02.140583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:02.140583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:02.140583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:02.140583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:02.140583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:02.141879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:02.141879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:02.141879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:02.141879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:02.141879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:02.141879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:02.141879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:02.141879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:02.141879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:02.141879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:02.141879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:02.141879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:02.141879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:02.141879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:02.141879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:02.141879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:02.142691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:02.142691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:02.142691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:02.142691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:02.142691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:02.142691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:02.142691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:02.142691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:02.142691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:02.142691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:02.142691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:54:02.142691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:02.143881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf306768-fa65-417e-b292-c77b342171fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:02.148750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf306768-fa65-417e-b292-c77b342171fb None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:02.148855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 175/341] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:54:01 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 333 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:02.151674 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841e1f1-248e-4469-8cbe-01af2093cb44 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:02.332671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3841e1f1-248e-4469-8cbe-01af2093cb44 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:02.382704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 167/342] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 639 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 15:54:02.384344 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f48b8-95c4-49af-b5d3-8c83f93fa7b2 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:02.384795 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f48b8-95c4-49af-b5d3-8c83f93fa7b2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:02.385214 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30f48b8-95c4-49af-b5d3-8c83f93fa7b2 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:02.386623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 168/343] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:02.392113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7038d67-5530-41d1-93f5-da8e72a3f86d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:02.392641 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7038d67-5530-41d1-93f5-da8e72a3f86d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:02.393081 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7038d67-5530-41d1-93f5-da8e72a3f86d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:02.404226 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81075066-87cd-44a4-9dbf-b35bc0e16ad0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:02.404923 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81075066-87cd-44a4-9dbf-b35bc0e16ad0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:02.405223 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81075066-87cd-44a4-9dbf-b35bc0e16ad0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:02.405607 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81075066-87cd-44a4-9dbf-b35bc0e16ad0 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:02.406344 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81075066-87cd-44a4-9dbf-b35bc0e16ad0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:02.406668 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81075066-87cd-44a4-9dbf-b35bc0e16ad0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:02.408382 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81075066-87cd-44a4-9dbf-b35bc0e16ad0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:02.414249 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f7038d67-5530-41d1-93f5-da8e72a3f86d None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:02.417281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f7038d67-5530-41d1-93f5-da8e72a3f86d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:02.423567 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 176/344] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 272 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:02.424548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81075066-87cd-44a4-9dbf-b35bc0e16ad0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:02.476483 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:02.492213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:02.522175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 169/345] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:54:02.522841 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7038d67-5530-41d1-93f5-da8e72a3f86d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:02.552164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81075066-87cd-44a4-9dbf-b35bc0e16ad0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:02.588952 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a2f422-35b4-41eb-9d78-4cb0a1ca7ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:02.589759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2f422-35b4-41eb-9d78-4cb0a1ca7ca1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:02.590041 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2f422-35b4-41eb-9d78-4cb0a1ca7ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:02.590316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a2f422-35b4-41eb-9d78-4cb0a1ca7ca1 None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:02.591505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2f422-35b4-41eb-9d78-4cb0a1ca7ca1 None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:02.593205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a2f422-35b4-41eb-9d78-4cb0a1ca7ca1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:02.594902 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 177/346] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:54:02 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:02.596164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94751b6-2bf1-4b1c-94f4-9d0db3e0b9ff None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:02.596418 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94751b6-2bf1-4b1c-94f4-9d0db3e0b9ff None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:02.596693 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94751b6-2bf1-4b1c-94f4-9d0db3e0b9ff None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:02.598191 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 178/347] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:02.598887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94751b6-2bf1-4b1c-94f4-9d0db3e0b9ff None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:02.637047 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7038d67-5530-41d1-93f5-da8e72a3f86d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:02.675777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6617028-4b21-4fc5-9e49-6ec19b086cc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:02.677151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6617028-4b21-4fc5-9e49-6ec19b086cc6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:02.677151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6617028-4b21-4fc5-9e49-6ec19b086cc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:02.677151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6617028-4b21-4fc5-9e49-6ec19b086cc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:02.702639 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:02.706450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94751b6-2bf1-4b1c-94f4-9d0db3e0b9ff None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:02.728220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:02.751365 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9cf35-a170-4a7c-94a2-b21e1d9ad617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:02.752680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9cf35-a170-4a7c-94a2-b21e1d9ad617 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:02.752965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9cf35-a170-4a7c-94a2-b21e1d9ad617 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:02.753241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b9cf35-a170-4a7c-94a2-b21e1d9ad617 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:02.764126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6617028-4b21-4fc5-9e49-6ec19b086cc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:02.765689 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6617028-4b21-4fc5-9e49-6ec19b086cc6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:02.770896 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b9cf35-a170-4a7c-94a2-b21e1d9ad617 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:02.780746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-68b9cf35-a170-4a7c-94a2-b21e1d9ad617 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:54:02.780746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:02.780746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:02.780746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:02.780746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:02.780746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:02.780746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:02.780746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:02.780746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:02.780746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:02.780746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:02.780746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:02.780746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:02.780746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:02.780746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:02.780746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:02.781723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:02.781723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:02.781723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:02.781723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:02.781723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:02.781723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:02.781723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:02.781723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:02.781723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:02.781723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:02.781723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:02.781723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:02.781723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:02.781723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:02.781723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:02.781723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:02.782636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:02.782636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:02.782636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:02.782636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:02.782636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:02.782636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:02.782636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:02.782636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:02.782636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:02.782636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:02.782636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. May 25 15:54:02.782636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:02.785668 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 179/348] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:02.786850 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9cf35-a170-4a7c-94a2-b21e1d9ad617 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:02.876631 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 170/349] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:02 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:54:02.878646 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed94f6-4d26-406e-bb22-6fd79ed659db None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:02.878646 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed94f6-4d26-406e-bb22-6fd79ed659db None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:02.878873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ed94f6-4d26-406e-bb22-6fd79ed659db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:02.896501 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f0ed94f6-4d26-406e-bb22-6fd79ed659db None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:02.897835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f0ed94f6-4d26-406e-bb22-6fd79ed659db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:02.907735 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b9cf35-a170-4a7c-94a2-b21e1d9ad617 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:02.944861 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865cd602-3bea-44ab-a3d7-87c6dcd2a329 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:02.946105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cd602-3bea-44ab-a3d7-87c6dcd2a329 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:02.946534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cd602-3bea-44ab-a3d7-87c6dcd2a329 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:02.947053 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-865cd602-3bea-44ab-a3d7-87c6dcd2a329 None admin] PATH_INFO: `/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:02.953965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865cd602-3bea-44ab-a3d7-87c6dcd2a329 None admin] RBAC: Authorizing `identity:get_group(group_id=a9d83e0a3baf472e90de2b6ec1a6f6f4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:02.955882 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-865cd602-3bea-44ab-a3d7-87c6dcd2a329 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:02.958158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 180/350] 10.222.0.85 () {56 vars in 1180 bytes} [Wed May 25 15:54:02 2022] GET /identity/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:02.959154 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865cd602-3bea-44ab-a3d7-87c6dcd2a329 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:02.973736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 171/351] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:02.974456 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed94f6-4d26-406e-bb22-6fd79ed659db None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:02.982842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:02.995514 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:03.083700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ed94f6-4d26-406e-bb22-6fd79ed659db None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:03.100010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-865cd602-3bea-44ab-a3d7-87c6dcd2a329 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:03.119564 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7e930-59c8-459a-b801-a1345f14d45d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:03.120465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7e930-59c8-459a-b801-a1345f14d45d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:03.120749 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7e930-59c8-459a-b801-a1345f14d45d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:03.121021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c7e930-59c8-459a-b801-a1345f14d45d None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:03.122215 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7e930-59c8-459a-b801-a1345f14d45d None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:03.123683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c7e930-59c8-459a-b801-a1345f14d45d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:03.125734 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 172/352] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:54:02 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:03.126956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7e930-59c8-459a-b801-a1345f14d45d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:03.147247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf663f-c461-4f45-9a85-68feb0f05f6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:03.148267 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf663f-c461-4f45-9a85-68feb0f05f6d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:03.148561 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf663f-c461-4f45-9a85-68feb0f05f6d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:03.148846 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bf663f-c461-4f45-9a85-68feb0f05f6d None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:03.149612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf663f-c461-4f45-9a85-68feb0f05f6d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:03.149932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf663f-c461-4f45-9a85-68feb0f05f6d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:03.151783 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1bf663f-c461-4f45-9a85-68feb0f05f6d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:03.163085 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 181/353] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:54:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:03.163992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf663f-c461-4f45-9a85-68feb0f05f6d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:03.210536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:03.245740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c7e930-59c8-459a-b801-a1345f14d45d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:03.272623 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:03.289671 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ceeee7-f8ae-441e-b0bd-526cade1f4ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:03.290476 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ceeee7-f8ae-441e-b0bd-526cade1f4ff None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:03.290771 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bf663f-c461-4f45-9a85-68feb0f05f6d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:03.290889 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ceeee7-f8ae-441e-b0bd-526cade1f4ff None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:03.291233 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ceeee7-f8ae-441e-b0bd-526cade1f4ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:03.294803 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ceeee7-f8ae-441e-b0bd-526cade1f4ff None admin] RBAC: Adding query filter params (group.id=a9d83e0a3baf472e90de2b6ec1a6f6f4, role.id=9c81542451e54c0ea22a20c026be6be7, scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:03.295196 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ceeee7-f8ae-441e-b0bd-526cade1f4ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:03.296816 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ceeee7-f8ae-441e-b0bd-526cade1f4ff None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:03.305005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 173/354] 10.222.0.85 () {56 vars in 1376 bytes} [Wed May 25 15:54:03 2022] GET /identity/v3/role_assignments?group.id=a9d83e0a3baf472e90de2b6ec1a6f6f4&scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d&role.id=9c81542451e54c0ea22a20c026be6be7 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:03.305540 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964621-2592-497b-a66d-1a56fb26c39e None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:03.311844 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964621-2592-497b-a66d-1a56fb26c39e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:03.311844 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed964621-2592-497b-a66d-1a56fb26c39e None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:03.311844 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 174/355] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:03.311844 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed964621-2592-497b-a66d-1a56fb26c39e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:03.347308 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4c6ba-6b91-412b-a680-773480434d82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:03.348212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c6ba-6b91-412b-a680-773480434d82 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:03.348514 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c6ba-6b91-412b-a680-773480434d82 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:03.348934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db4c6ba-6b91-412b-a680-773480434d82 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:03.365098 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db4c6ba-6b91-412b-a680-773480434d82 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:03.375102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-1db4c6ba-6b91-412b-a680-773480434d82 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:54:03.375102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:03.375102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:03.375102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:03.375102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:03.375102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:03.375102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:03.375102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:03.375102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:03.375102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:03.375102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:03.375102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:03.375102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:03.375102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:03.375102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:03.375102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:03.376134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:03.376134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:03.376134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:03.376134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:03.376134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:03.376134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:03.376134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:03.376134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:03.376134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:03.376134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:03.376134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:03.376134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:03.376134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:03.376134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:03.376134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:03.376134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:03.376929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:03.376929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:03.376929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:03.376929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:03.376929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:03.376929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:03.376929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:03.376929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:03.376929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:03.376929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:03.376929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:54:03.376929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:03.376929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 182/356] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:03 2022] GET /identity/v3/roles/member => generated 84 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:03.378156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4c6ba-6b91-412b-a680-773480434d82 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:03.465160 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed964621-2592-497b-a66d-1a56fb26c39e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:03.508703 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae2282f7-5b6c-4fc8-ae55-9187d00bb708 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:03.509642 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2282f7-5b6c-4fc8-ae55-9187d00bb708 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:03.510079 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2282f7-5b6c-4fc8-ae55-9187d00bb708 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:03.510364 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae2282f7-5b6c-4fc8-ae55-9187d00bb708 None admin] PATH_INFO: `/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:03.512258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2282f7-5b6c-4fc8-ae55-9187d00bb708 None admin] RBAC: Authorizing `identity:get_group(group_id=a9d83e0a3baf472e90de2b6ec1a6f6f4)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:03.514011 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae2282f7-5b6c-4fc8-ae55-9187d00bb708 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:03.515935 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 175/357] 10.222.0.85 () {56 vars in 1180 bytes} [Wed May 25 15:54:03 2022] GET /identity/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4 => generated 226 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:03.517213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4dcfd-3f2b-4e91-88ae-30b3617dcff9 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:03.517500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4dcfd-3f2b-4e91-88ae-30b3617dcff9 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:03.517777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c4dcfd-3f2b-4e91-88ae-30b3617dcff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:03.525680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:03.527333 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db4c6ba-6b91-412b-a680-773480434d82 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:03.541379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-88c4dcfd-3f2b-4e91-88ae-30b3617dcff9 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:03.542738 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-88c4dcfd-3f2b-4e91-88ae-30b3617dcff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:03.573606 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e84cfd-92e2-4e25-89ad-428e577c2d26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:03.574552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e84cfd-92e2-4e25-89ad-428e577c2d26 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:03.574801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e84cfd-92e2-4e25-89ad-428e577c2d26 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:03.575087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e84cfd-92e2-4e25-89ad-428e577c2d26 None admin] PATH_INFO: `/v3/users/1ab68bdb0bce44c6b7a7c995da8f87eb` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:03.576328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e84cfd-92e2-4e25-89ad-428e577c2d26 None admin] RBAC: Authorizing `identity:get_user(user_id=1ab68bdb0bce44c6b7a7c995da8f87eb)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:03.577886 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e84cfd-92e2-4e25-89ad-428e577c2d26 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:03.579711 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 183/358] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:54:03 2022] GET /identity/v3/users/1ab68bdb0bce44c6b7a7c995da8f87eb => generated 293 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:03.580423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e84cfd-92e2-4e25-89ad-428e577c2d26 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:03.597857 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:03.622977 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 176/359] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:03.623664 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4dcfd-3f2b-4e91-88ae-30b3617dcff9 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:03.702125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e84cfd-92e2-4e25-89ad-428e577c2d26 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:03.746373 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c4dcfd-3f2b-4e91-88ae-30b3617dcff9 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:03.750781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ef5eae-963a-49f1-a877-b5d086bcb9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:03.751478 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef5eae-963a-49f1-a877-b5d086bcb9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:03.751765 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef5eae-963a-49f1-a877-b5d086bcb9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:03.752076 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ef5eae-963a-49f1-a877-b5d086bcb9d3 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:03.752843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef5eae-963a-49f1-a877-b5d086bcb9d3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:03.753159 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef5eae-963a-49f1-a877-b5d086bcb9d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:03.754605 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ef5eae-963a-49f1-a877-b5d086bcb9d3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:03.764225 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 184/360] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:54:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:03.765586 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dfda7-b9e4-410c-8e19-f918d3f72a56 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:03.765881 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dfda7-b9e4-410c-8e19-f918d3f72a56 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:03.766167 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6dfda7-b9e4-410c-8e19-f918d3f72a56 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:03.767549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 185/361] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:03.768107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6dfda7-b9e4-410c-8e19-f918d3f72a56 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:03.790252 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:03.794732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a5a7d-c0be-4f46-b2ac-515de32c46fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:03.795414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a5a7d-c0be-4f46-b2ac-515de32c46fd None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:03.796145 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a5a7d-c0be-4f46-b2ac-515de32c46fd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:03.796703 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-987a5a7d-c0be-4f46-b2ac-515de32c46fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:03.797565 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a5a7d-c0be-4f46-b2ac-515de32c46fd None admin] RBAC: Adding query filter params (group.id=a9d83e0a3baf472e90de2b6ec1a6f6f4, role.id=1cc9b5d798c648c986537d006320581f, scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:03.799122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a5a7d-c0be-4f46-b2ac-515de32c46fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:03.800402 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-987a5a7d-c0be-4f46-b2ac-515de32c46fd None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:03.806890 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 177/362] 10.222.0.85 () {56 vars in 1376 bytes} [Wed May 25 15:54:03 2022] GET /identity/v3/role_assignments?group.id=a9d83e0a3baf472e90de2b6ec1a6f6f4&scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d&role.id=1cc9b5d798c648c986537d006320581f => generated 260 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:03.809180 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a5a7d-c0be-4f46-b2ac-515de32c46fd None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:03.887097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:03.908416 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6dfda7-b9e4-410c-8e19-f918d3f72a56 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:03.933567 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-987a5a7d-c0be-4f46-b2ac-515de32c46fd None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:03.957862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019bc38-fe46-4525-a52b-6a2e41566e75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:03.958457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019bc38-fe46-4525-a52b-6a2e41566e75 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:03.958745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019bc38-fe46-4525-a52b-6a2e41566e75 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:03.959017 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0019bc38-fe46-4525-a52b-6a2e41566e75 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:03.961067 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0019bc38-fe46-4525-a52b-6a2e41566e75 None admin] RBAC: Adding query filter params (role.id=b7283d3771c94f458ebfcaef5784afa0, user.id=1ab68bdb0bce44c6b7a7c995da8f87eb, scope.system=all) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:03.961391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0019bc38-fe46-4525-a52b-6a2e41566e75 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:03.962891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0019bc38-fe46-4525-a52b-6a2e41566e75 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:03.987312 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 186/363] 10.222.0.85 () {56 vars in 1308 bytes} [Wed May 25 15:54:03 2022] GET /identity/v3/role_assignments?user.id=1ab68bdb0bce44c6b7a7c995da8f87eb&scope.system=all&role.id=b7283d3771c94f458ebfcaef5784afa0 => generated 510 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:03.988768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019bc38-fe46-4525-a52b-6a2e41566e75 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:03.993477 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9cf7b38-cd92-4e50-b40b-bba15d2f643c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:03.994334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf7b38-cd92-4e50-b40b-bba15d2f643c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:03.994622 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf7b38-cd92-4e50-b40b-bba15d2f643c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:03.994904 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9cf7b38-cd92-4e50-b40b-bba15d2f643c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:04.020484 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9cf7b38-cd92-4e50-b40b-bba15d2f643c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:04.031346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-c9cf7b38-cd92-4e50-b40b-bba15d2f643c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:54:04.031346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:04.031346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:04.031346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:04.031346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:04.031346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:04.031346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:04.031346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:04.031346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:04.031346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:04.031346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:04.031346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:04.031346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:04.031346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:04.031346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:04.031346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:04.032379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:04.032379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:04.032379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:04.032379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:04.032379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:04.032379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:04.032379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:04.032379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:04.032379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:04.032379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:04.032379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:04.032379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:04.032379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:04.032379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:04.032379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:04.032379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:04.033210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:04.033210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:04.033210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:04.033210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:04.033210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:04.033210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:04.033210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:04.033210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:04.033210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:04.033210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:04.033210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:54:04.033210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:04.036643 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 178/364] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:54:03 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:04.038393 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6d4ae-8c4d-4a1b-81d3-13f0d6e96514 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:04.038893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6d4ae-8c4d-4a1b-81d3-13f0d6e96514 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:04.039598 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d6d4ae-8c4d-4a1b-81d3-13f0d6e96514 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:04.076285 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d3d6d4ae-8c4d-4a1b-81d3-13f0d6e96514 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:04.077963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d3d6d4ae-8c4d-4a1b-81d3-13f0d6e96514 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:04.155130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0019bc38-fe46-4525-a52b-6a2e41566e75 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:04.158883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:04.201158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417c5c7-7658-46bb-b334-8027ad31fa33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:04.202234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417c5c7-7658-46bb-b334-8027ad31fa33 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:04.202804 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417c5c7-7658-46bb-b334-8027ad31fa33 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:04.203297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a417c5c7-7658-46bb-b334-8027ad31fa33 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:04.220075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 179/365] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:04.220926 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6d4ae-8c4d-4a1b-81d3-13f0d6e96514 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:04.239206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:04.297761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a417c5c7-7658-46bb-b334-8027ad31fa33 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:04.299261 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a417c5c7-7658-46bb-b334-8027ad31fa33 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:04.390620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d6d4ae-8c4d-4a1b-81d3-13f0d6e96514 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:04.435923 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 187/366] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 448 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:54:04.436793 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417c5c7-7658-46bb-b334-8027ad31fa33 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:04.444265 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e79b6d-6524-4058-8c2b-d9e9d4b14d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:04.444964 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e79b6d-6524-4058-8c2b-d9e9d4b14d70 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:04.445255 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e79b6d-6524-4058-8c2b-d9e9d4b14d70 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:04.445539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e79b6d-6524-4058-8c2b-d9e9d4b14d70 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:04.446308 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e79b6d-6524-4058-8c2b-d9e9d4b14d70 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:04.446624 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e79b6d-6524-4058-8c2b-d9e9d4b14d70 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:04.448512 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e79b6d-6524-4058-8c2b-d9e9d4b14d70 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:04.460826 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 180/367] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:04.461693 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e79b6d-6524-4058-8c2b-d9e9d4b14d70 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:04.480259 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:04.521622 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:04.565904 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a417c5c7-7658-46bb-b334-8027ad31fa33 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:04.590537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e79b6d-6524-4058-8c2b-d9e9d4b14d70 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:04.615981 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6becb718-9d5d-40c4-b458-9b978f3443f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:04.616905 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6becb718-9d5d-40c4-b458-9b978f3443f3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:04.617261 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6becb718-9d5d-40c4-b458-9b978f3443f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:04.617621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6becb718-9d5d-40c4-b458-9b978f3443f3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:04.629263 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6becb718-9d5d-40c4-b458-9b978f3443f3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:04.636717 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dee161-7165-470d-8f3f-cf65ab2cbaa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:04.637663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dee161-7165-470d-8f3f-cf65ab2cbaa2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:04.637956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dee161-7165-470d-8f3f-cf65ab2cbaa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:04.638238 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dee161-7165-470d-8f3f-cf65ab2cbaa2 None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:04.639100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-6becb718-9d5d-40c4-b458-9b978f3443f3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:04.639100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:04.639100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:04.639100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:04.639100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:04.639100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:04.639100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:04.639100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:04.639100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:04.639100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:04.639100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:04.639100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:04.639100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:04.639100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:04.639100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:04.639100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:04.640073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:04.640073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:04.640073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:04.640073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:04.640073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:04.640073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:04.640073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:04.640073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:04.640073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:04.640073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:04.640073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:04.640073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:04.640073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:04.640073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:04.640073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:04.640073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:04.640887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:04.640887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:04.640887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:04.640887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:04.640887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:04.640887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:04.640887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:04.640887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:04.640887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:04.640887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:04.640887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:04.640887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:04.641517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dee161-7165-470d-8f3f-cf65ab2cbaa2 None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:04.641586 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 188/368] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:54:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:04.641650 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2dee161-7165-470d-8f3f-cf65ab2cbaa2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:04.642204 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6becb718-9d5d-40c4-b458-9b978f3443f3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:04.644631 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 181/369] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:54:04 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:04.648880 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dee161-7165-470d-8f3f-cf65ab2cbaa2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:04.767188 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:04.774521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6becb718-9d5d-40c4-b458-9b978f3443f3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:04.779202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dee161-7165-470d-8f3f-cf65ab2cbaa2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:04.800333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:04.819548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa6506-4920-4cf3-971d-e4fab3bdc61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:04.820367 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa6506-4920-4cf3-971d-e4fab3bdc61d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:04.820725 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa6506-4920-4cf3-971d-e4fab3bdc61d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:04.821081 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fa6506-4920-4cf3-971d-e4fab3bdc61d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:04.824151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c0961-d109-4112-92b2-c763ce386492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:04.825754 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c0961-d109-4112-92b2-c763ce386492 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:04.826006 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c0961-d109-4112-92b2-c763ce386492 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:04.826289 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745c0961-d109-4112-92b2-c763ce386492 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:04.827082 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c0961-d109-4112-92b2-c763ce386492 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:04.827441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c0961-d109-4112-92b2-c763ce386492 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:04.828918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745c0961-d109-4112-92b2-c763ce386492 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:04.838350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 182/370] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:04.839502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c0961-d109-4112-92b2-c763ce386492 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:04.912648 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa6506-4920-4cf3-971d-e4fab3bdc61d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:04.915000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fa6506-4920-4cf3-971d-e4fab3bdc61d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:04.958648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745c0961-d109-4112-92b2-c763ce386492 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:04.996233 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc10261-eb47-4d50-a721-51c2052fa083 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:04.997258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc10261-eb47-4d50-a721-51c2052fa083 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:04.997653 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc10261-eb47-4d50-a721-51c2052fa083 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:04.998027 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afc10261-eb47-4d50-a721-51c2052fa083 None admin] PATH_INFO: `/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:04.999459 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc10261-eb47-4d50-a721-51c2052fa083 None admin] RBAC: Authorizing `identity:get_group(group_id=a9d83e0a3baf472e90de2b6ec1a6f6f4)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:05.001012 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afc10261-eb47-4d50-a721-51c2052fa083 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:05.003775 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 183/371] 10.222.0.85 () {56 vars in 1180 bytes} [Wed May 25 15:54:04 2022] GET /identity/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4 => generated 226 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:05.004424 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc10261-eb47-4d50-a721-51c2052fa083 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:05.031988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:05.047043 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 189/372] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:04 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:54:05.047043 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa6506-4920-4cf3-971d-e4fab3bdc61d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:05.070528 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:05.141252 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afc10261-eb47-4d50-a721-51c2052fa083 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:05.178890 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fa6506-4920-4cf3-971d-e4fab3bdc61d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:05.183533 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a85d7f-703b-4868-8555-9fee45a5fcd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:05.184234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a85d7f-703b-4868-8555-9fee45a5fcd4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:05.184528 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a85d7f-703b-4868-8555-9fee45a5fcd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:05.184814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a85d7f-703b-4868-8555-9fee45a5fcd4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:05.226667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479b5c7-0c34-493d-abd5-8d781f896933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:05.227336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479b5c7-0c34-493d-abd5-8d781f896933 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:05.227623 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479b5c7-0c34-493d-abd5-8d781f896933 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:05.227927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479b5c7-0c34-493d-abd5-8d781f896933 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:05.228771 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479b5c7-0c34-493d-abd5-8d781f896933 None admin] RBAC: Adding query filter params (group.id=a9d83e0a3baf472e90de2b6ec1a6f6f4, role.id=9c81542451e54c0ea22a20c026be6be7, scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:05.229086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479b5c7-0c34-493d-abd5-8d781f896933 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:05.230651 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479b5c7-0c34-493d-abd5-8d781f896933 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:05.236678 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 190/373] 10.222.0.85 () {56 vars in 1376 bytes} [Wed May 25 15:54:05 2022] GET /identity/v3/role_assignments?group.id=a9d83e0a3baf472e90de2b6ec1a6f6f4&scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7&role.id=9c81542451e54c0ea22a20c026be6be7 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:05.237755 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479b5c7-0c34-493d-abd5-8d781f896933 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:05.275536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a85d7f-703b-4868-8555-9fee45a5fcd4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:05.277974 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a85d7f-703b-4868-8555-9fee45a5fcd4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:05.315381 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:05.336467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:05.419741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479b5c7-0c34-493d-abd5-8d781f896933 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:05.457366 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 184/374] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:05 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 452 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:54:05.464247 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a85d7f-703b-4868-8555-9fee45a5fcd4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:05.479336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505ce36-bacb-44bd-a477-ec3bd15efcdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:05.480347 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505ce36-bacb-44bd-a477-ec3bd15efcdf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:05.480641 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505ce36-bacb-44bd-a477-ec3bd15efcdf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:05.480927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3505ce36-bacb-44bd-a477-ec3bd15efcdf None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:05.482330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505ce36-bacb-44bd-a477-ec3bd15efcdf None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:05.491866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3505ce36-bacb-44bd-a477-ec3bd15efcdf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:05.491866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 191/375] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:54:05 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:05.494139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505ce36-bacb-44bd-a477-ec3bd15efcdf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:05.603551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3505ce36-bacb-44bd-a477-ec3bd15efcdf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:05.630789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:05.640622 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b41ebe-a146-45d4-b8ac-6a1781ff0784 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:05.641727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b41ebe-a146-45d4-b8ac-6a1781ff0784 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:05.642180 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b41ebe-a146-45d4-b8ac-6a1781ff0784 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:05.642608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b41ebe-a146-45d4-b8ac-6a1781ff0784 None admin] PATH_INFO: `/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:05.644046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b41ebe-a146-45d4-b8ac-6a1781ff0784 None admin] RBAC: Authorizing `identity:get_group(group_id=a9d83e0a3baf472e90de2b6ec1a6f6f4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:05.645869 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b41ebe-a146-45d4-b8ac-6a1781ff0784 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:05.648113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 192/376] 10.222.0.85 () {56 vars in 1180 bytes} [Wed May 25 15:54:05 2022] GET /identity/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4 => generated 226 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:05.648247 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a85d7f-703b-4868-8555-9fee45a5fcd4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:05.656551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b41ebe-a146-45d4-b8ac-6a1781ff0784 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:05.717960 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3d753-3a13-4a16-a87a-194c101bb160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:05.718956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3d753-3a13-4a16-a87a-194c101bb160 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:05.719242 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3d753-3a13-4a16-a87a-194c101bb160 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:05.719525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3d753-3a13-4a16-a87a-194c101bb160 None admin] PATH_INFO: `/v3/projects/12da861032034c60b092c703b1fe5e90` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:05.720882 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d3d753-3a13-4a16-a87a-194c101bb160 None admin] RBAC: Authorizing `identity:get_project(project_id=12da861032034c60b092c703b1fe5e90)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:05.722507 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d3d753-3a13-4a16-a87a-194c101bb160 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:05.724774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 185/377] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:54:05 2022] GET /identity/v3/projects/12da861032034c60b092c703b1fe5e90 => generated 344 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:05.730166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3d753-3a13-4a16-a87a-194c101bb160 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:05.765461 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:05.772307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b41ebe-a146-45d4-b8ac-6a1781ff0784 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:05.809559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-195a9141-ce14-450c-9f94-6f545161f503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:05.810803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a9141-ce14-450c-9f94-6f545161f503 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:05.811462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a9141-ce14-450c-9f94-6f545161f503 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:05.812288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-195a9141-ce14-450c-9f94-6f545161f503 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:05.813954 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a9141-ce14-450c-9f94-6f545161f503 None admin] RBAC: Adding query filter params (group.id=a9d83e0a3baf472e90de2b6ec1a6f6f4, role.id=1cc9b5d798c648c986537d006320581f, scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:05.814767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a9141-ce14-450c-9f94-6f545161f503 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:05.816684 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-195a9141-ce14-450c-9f94-6f545161f503 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:05.823393 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 193/378] 10.222.0.85 () {56 vars in 1376 bytes} [Wed May 25 15:54:05 2022] GET /identity/v3/role_assignments?group.id=a9d83e0a3baf472e90de2b6ec1a6f6f4&scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7&role.id=1cc9b5d798c648c986537d006320581f => generated 260 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:05.926964 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3d753-3a13-4a16-a87a-194c101bb160 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:05.957798 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5f482-3369-4b11-a59c-dfbda8c04fad None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:05.958093 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5f482-3369-4b11-a59c-dfbda8c04fad None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:05.958376 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af5f482-3369-4b11-a59c-dfbda8c04fad None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:05.963537 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 194/379] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:05 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:05.969935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c70a6-6d1b-4fd0-9616-e354373add12 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:05.970244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c70a6-6d1b-4fd0-9616-e354373add12 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:05.970507 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0c70a6-6d1b-4fd0-9616-e354373add12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:05.979058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a014c6-dd70-4ac5-b8a2-af81f23791fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:05.986011 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a014c6-dd70-4ac5-b8a2-af81f23791fb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:05.986011 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a014c6-dd70-4ac5-b8a2-af81f23791fb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:05.986426 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a014c6-dd70-4ac5-b8a2-af81f23791fb None admin] PATH_INFO: `/v3/groups/6b5136a4c5994d6d824c7324ea0487a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:06.000373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-da0c70a6-6d1b-4fd0-9616-e354373add12 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:06.001609 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-da0c70a6-6d1b-4fd0-9616-e354373add12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:06.004532 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a014c6-dd70-4ac5-b8a2-af81f23791fb None admin] RBAC: Authorizing `identity:get_group(group_id=6b5136a4c5994d6d824c7324ea0487a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:06.006039 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a014c6-dd70-4ac5-b8a2-af81f23791fb None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:06.007908 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 186/380] 10.222.0.85 () {56 vars in 1180 bytes} [Wed May 25 15:54:05 2022] GET /identity/v3/groups/6b5136a4c5994d6d824c7324ea0487a1 => generated 229 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:06.015306 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a014c6-dd70-4ac5-b8a2-af81f23791fb None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:06.113368 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:06.138676 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 195/381] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:54:06.158328 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:06.162591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c70a6-6d1b-4fd0-9616-e354373add12 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:06.185348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a014c6-dd70-4ac5-b8a2-af81f23791fb None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:06.231856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79c9c0-5319-4bcf-b5eb-f8ec82a4c707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:06.232572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79c9c0-5319-4bcf-b5eb-f8ec82a4c707 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:06.232856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79c9c0-5319-4bcf-b5eb-f8ec82a4c707 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:06.233135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee79c9c0-5319-4bcf-b5eb-f8ec82a4c707 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:06.234023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79c9c0-5319-4bcf-b5eb-f8ec82a4c707 None admin] RBAC: Adding query filter params (group.id=6b5136a4c5994d6d824c7324ea0487a1, role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=12da861032034c60b092c703b1fe5e90) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:06.234342 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79c9c0-5319-4bcf-b5eb-f8ec82a4c707 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:06.238988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee79c9c0-5319-4bcf-b5eb-f8ec82a4c707 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:06.252637 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 187/382] 10.222.0.85 () {56 vars in 1376 bytes} [Wed May 25 15:54:06 2022] GET /identity/v3/role_assignments?group.id=6b5136a4c5994d6d824c7324ea0487a1&scope.project.id=12da861032034c60b092c703b1fe5e90&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 260 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:06.359691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0c70a6-6d1b-4fd0-9616-e354373add12 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:06.420804 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c7a4a16-0a8f-4b66-bc6b-35a266f64ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:06.421733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a4a16-0a8f-4b66-bc6b-35a266f64ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:06.422019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a4a16-0a8f-4b66-bc6b-35a266f64ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:06.422298 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c7a4a16-0a8f-4b66-bc6b-35a266f64ab7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:06.434662 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c7a4a16-0a8f-4b66-bc6b-35a266f64ab7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:06.443200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-6c7a4a16-0a8f-4b66-bc6b-35a266f64ab7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:54:06.443200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:06.443200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:06.443200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:06.443200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:06.443200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:06.443200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:06.443200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:06.443200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:06.443200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:06.443200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:06.443200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:06.443200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:06.443200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:06.443200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:06.443200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:06.444197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:06.444197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:06.444197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:06.444197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:06.444197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:06.444197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:06.444197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:06.444197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:06.444197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:06.444197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:06.444197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:06.444197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:06.444197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:06.444197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:06.444197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:06.444197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:06.445010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:06.445010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:06.445010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:06.445010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:06.445010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:06.445010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:06.445010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:06.445010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:06.445010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:06.445010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:06.445010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:54:06.445010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:06.445010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 196/383] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:54:06.450535 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79c9c0-5319-4bcf-b5eb-f8ec82a4c707 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:06.583976 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee79c9c0-5319-4bcf-b5eb-f8ec82a4c707 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:06.591269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad5535-c84c-417e-a572-a614a130dcd4 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:06.591557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad5535-c84c-417e-a572-a614a130dcd4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:06.591864 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acad5535-c84c-417e-a572-a614a130dcd4 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:06.593542 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 197/384] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:06.613417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832afd2a-1f58-4817-8c2f-0a87d9c5a66c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:06.613721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832afd2a-1f58-4817-8c2f-0a87d9c5a66c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:06.613985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-832afd2a-1f58-4817-8c2f-0a87d9c5a66c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:06.630077 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d4efa-2286-4ef0-acf7-eb37769a6a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:06.631170 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d4efa-2286-4ef0-acf7-eb37769a6a55 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:06.631658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d4efa-2286-4ef0-acf7-eb37769a6a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:06.632178 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656d4efa-2286-4ef0-acf7-eb37769a6a55 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:06.633150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d4efa-2286-4ef0-acf7-eb37769a6a55 None admin] RBAC: Adding query filter params (name=member) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:06.633682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d4efa-2286-4ef0-acf7-eb37769a6a55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:06.635283 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-656d4efa-2286-4ef0-acf7-eb37769a6a55 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:06.645227 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 188/385] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:54:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:06.655865 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-832afd2a-1f58-4817-8c2f-0a87d9c5a66c None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:06.655865 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-832afd2a-1f58-4817-8c2f-0a87d9c5a66c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:06.657446 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d4efa-2286-4ef0-acf7-eb37769a6a55 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:06.676783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:06.720428 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:06.749385 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 198/386] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:06.796755 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832afd2a-1f58-4817-8c2f-0a87d9c5a66c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:06.845595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656d4efa-2286-4ef0-acf7-eb37769a6a55 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:06.897828 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf573fa3-ad7e-4a4d-8a78-dc63a2273ed9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:06.898723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf573fa3-ad7e-4a4d-8a78-dc63a2273ed9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:06.899008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf573fa3-ad7e-4a4d-8a78-dc63a2273ed9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:06.899287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf573fa3-ad7e-4a4d-8a78-dc63a2273ed9 None admin] PATH_INFO: `/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:06.900649 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf573fa3-ad7e-4a4d-8a78-dc63a2273ed9 None admin] RBAC: Authorizing `identity:get_group(group_id=a9d83e0a3baf472e90de2b6ec1a6f6f4)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:06.902282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf573fa3-ad7e-4a4d-8a78-dc63a2273ed9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:06.904238 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 189/387] 10.222.0.85 () {56 vars in 1180 bytes} [Wed May 25 15:54:06 2022] GET /identity/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4 => generated 226 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:06.909944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf573fa3-ad7e-4a4d-8a78-dc63a2273ed9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:07.011587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-832afd2a-1f58-4817-8c2f-0a87d9c5a66c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:07.013218 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:07.047957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf573fa3-ad7e-4a4d-8a78-dc63a2273ed9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:07.083472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cedf632-3ee3-4059-96c8-1b7e1c4230b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:07.083718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476c297-9e6c-4649-9cc2-b94b3dc58060 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:07.087845 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476c297-9e6c-4649-9cc2-b94b3dc58060 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:07.087958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedf632-3ee3-4059-96c8-1b7e1c4230b5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:07.088034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476c297-9e6c-4649-9cc2-b94b3dc58060 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:07.088121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedf632-3ee3-4059-96c8-1b7e1c4230b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:07.088203 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476c297-9e6c-4649-9cc2-b94b3dc58060 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:07.088273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cedf632-3ee3-4059-96c8-1b7e1c4230b5 None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:07.088273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cedf632-3ee3-4059-96c8-1b7e1c4230b5 None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:07.088273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cedf632-3ee3-4059-96c8-1b7e1c4230b5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:07.090133 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 190/388] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:54:06 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:07.095874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cedf632-3ee3-4059-96c8-1b7e1c4230b5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:07.120743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e476c297-9e6c-4649-9cc2-b94b3dc58060 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:07.129125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e476c297-9e6c-4649-9cc2-b94b3dc58060 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:54:07.129125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:07.129125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:07.129125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:07.129125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:07.129125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:07.129125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:07.129125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:07.129125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:07.129125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:07.129125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:07.129125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:07.129125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:07.129125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:07.129125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:07.129125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:07.130355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:07.130355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:07.130355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:07.130355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:07.130355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:07.130355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:07.130355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:07.130355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:07.130355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:07.130355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:07.130355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:07.130355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:07.130355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:07.130355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:07.130355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:07.130355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:07.131357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:07.131357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:07.131357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:07.131357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:07.131357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:07.131357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:07.131357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:07.131357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:07.131357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:07.131357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:07.131357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:54:07.131357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:07.131357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 199/389] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:54:06 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:54:07.138667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476c297-9e6c-4649-9cc2-b94b3dc58060 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:07.203145 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:07.222241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cedf632-3ee3-4059-96c8-1b7e1c4230b5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:07.275070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476c297-9e6c-4649-9cc2-b94b3dc58060 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:07.288560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e3d2a-3ee5-47c7-8a8c-a8b2157e2c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:07.289487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e3d2a-3ee5-47c7-8a8c-a8b2157e2c06 None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:07.289779 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e3d2a-3ee5-47c7-8a8c-a8b2157e2c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:07.290067 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2e3d2a-3ee5-47c7-8a8c-a8b2157e2c06 None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:07.301435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e3d2a-3ee5-47c7-8a8c-a8b2157e2c06 None admin] RBAC: Authorizing `identity:create_grant(project_id=89a647c5dfc34a43ad12c49f8f57d99d, group_id=a9d83e0a3baf472e90de2b6ec1a6f6f4, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:07.302214 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c2e3d2a-3ee5-47c7-8a8c-a8b2157e2c06 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:07.325639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:07.342482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 191/390] 10.222.0.85 () {58 vars in 1442 bytes} [Wed May 25 15:54:07 2022] PUT /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:54:07.353619 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc4678-102c-44e3-95fa-befce5837a29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:07.354328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc4678-102c-44e3-95fa-befce5837a29 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:07.354623 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc4678-102c-44e3-95fa-befce5837a29 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:07.354904 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cbc4678-102c-44e3-95fa-befce5837a29 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:07.355670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc4678-102c-44e3-95fa-befce5837a29 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:07.356011 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc4678-102c-44e3-95fa-befce5837a29 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:07.357423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cbc4678-102c-44e3-95fa-befce5837a29 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:07.375781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 200/391] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:07 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:07.382291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e3d2a-3ee5-47c7-8a8c-a8b2157e2c06 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:07.573236 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c2e3d2a-3ee5-47c7-8a8c-a8b2157e2c06 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:07.621543 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ead9c4-c0ee-4c16-a288-7278dd852162 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:07.622429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ead9c4-c0ee-4c16-a288-7278dd852162 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:07.622716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ead9c4-c0ee-4c16-a288-7278dd852162 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:07.622996 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08ead9c4-c0ee-4c16-a288-7278dd852162 None admin] PATH_INFO: `/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:07.628551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ead9c4-c0ee-4c16-a288-7278dd852162 None admin] RBAC: Authorizing `identity:get_group(group_id=a9d83e0a3baf472e90de2b6ec1a6f6f4)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:07.630221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08ead9c4-c0ee-4c16-a288-7278dd852162 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:07.632446 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 192/392] 10.222.0.85 () {56 vars in 1180 bytes} [Wed May 25 15:54:07 2022] GET /identity/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4 => generated 226 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:07.638175 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc4678-102c-44e3-95fa-befce5837a29 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:07.741767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:07.745695 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cbc4678-102c-44e3-95fa-befce5837a29 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:07.782534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14039f71-897c-4d70-8031-9395f5cc877b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:07.783856 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14039f71-897c-4d70-8031-9395f5cc877b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:07.784384 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14039f71-897c-4d70-8031-9395f5cc877b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:07.784858 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14039f71-897c-4d70-8031-9395f5cc877b None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:07.786262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14039f71-897c-4d70-8031-9395f5cc877b None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:07.787951 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14039f71-897c-4d70-8031-9395f5cc877b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:07.790172 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 201/393] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:54:07 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:07.796193 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ead9c4-c0ee-4c16-a288-7278dd852162 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:07.867108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:07.907906 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08ead9c4-c0ee-4c16-a288-7278dd852162 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:07.947602 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcc0993-9873-4b62-9d67-1c4ba5ad058a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:07.948927 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc0993-9873-4b62-9d67-1c4ba5ad058a None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:07.949407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc0993-9873-4b62-9d67-1c4ba5ad058a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:07.949876 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcc0993-9873-4b62-9d67-1c4ba5ad058a None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4/roles/1cc9b5d798c648c986537d006320581f` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:07.952388 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc0993-9873-4b62-9d67-1c4ba5ad058a None admin] RBAC: Authorizing `identity:create_grant(project_id=89a647c5dfc34a43ad12c49f8f57d99d, group_id=a9d83e0a3baf472e90de2b6ec1a6f6f4, role_id=1cc9b5d798c648c986537d006320581f)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:07.954049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fcc0993-9873-4b62-9d67-1c4ba5ad058a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:07.984982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 193/394] 10.222.0.85 () {58 vars in 1442 bytes} [Wed May 25 15:54:07 2022] PUT /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4/roles/1cc9b5d798c648c986537d006320581f => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:54:08.110963 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88453712-d4c6-4290-b825-e905e1453e40 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:08.111612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88453712-d4c6-4290-b825-e905e1453e40 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:08.112146 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88453712-d4c6-4290-b825-e905e1453e40 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:08.114190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 202/395] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:08.119699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d063a8c-831d-4092-9ebd-ad3875b8e2b5 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:08.120271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d063a8c-831d-4092-9ebd-ad3875b8e2b5 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:08.120753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d063a8c-831d-4092-9ebd-ad3875b8e2b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:08.145686 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-5d063a8c-831d-4092-9ebd-ad3875b8e2b5 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:08.147255 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5d063a8c-831d-4092-9ebd-ad3875b8e2b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:08.235724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 194/396] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:08.266005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88453712-d4c6-4290-b825-e905e1453e40 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:08.391441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88453712-d4c6-4290-b825-e905e1453e40 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:08.428572 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15907183-8fc7-42f6-bc72-90f252bbee6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:08.429814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15907183-8fc7-42f6-bc72-90f252bbee6c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:08.430323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15907183-8fc7-42f6-bc72-90f252bbee6c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:08.430794 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15907183-8fc7-42f6-bc72-90f252bbee6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:08.442381 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15907183-8fc7-42f6-bc72-90f252bbee6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:08.450602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-15907183-8fc7-42f6-bc72-90f252bbee6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:54:08.450602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:08.450602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:08.450602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:08.450602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:08.450602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:08.450602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:08.450602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:08.450602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:08.450602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:08.450602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:08.450602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:08.450602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:08.450602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:08.450602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:08.450602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:08.451533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:08.451533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:08.451533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:08.451533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:08.451533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:08.451533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:08.451533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:08.451533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:08.451533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:08.451533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:08.451533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:08.451533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:08.451533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:08.451533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:08.451533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:08.451533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:08.452450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:08.452450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:08.452450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:08.452450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:08.452450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:08.452450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:08.452450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:08.452450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:08.452450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:08.452450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:08.452450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:54:08.452450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:08.455185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 203/397] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:08.461015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d063a8c-831d-4092-9ebd-ad3875b8e2b5 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:08.529566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:08.581536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d063a8c-831d-4092-9ebd-ad3875b8e2b5 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:08.621783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38594853-81e4-4bfc-8644-e3cb4aa902a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:08.622873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38594853-81e4-4bfc-8644-e3cb4aa902a6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:08.623358 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38594853-81e4-4bfc-8644-e3cb4aa902a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:08.623915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38594853-81e4-4bfc-8644-e3cb4aa902a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:08.624903 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38594853-81e4-4bfc-8644-e3cb4aa902a6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:08.625449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38594853-81e4-4bfc-8644-e3cb4aa902a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:08.627089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38594853-81e4-4bfc-8644-e3cb4aa902a6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:08.639977 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 195/398] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:54:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:08.645488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15907183-8fc7-42f6-bc72-90f252bbee6c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:08.678556 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:08.783778 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15907183-8fc7-42f6-bc72-90f252bbee6c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:08.823806 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cd3db-be07-4dd8-ab61-c2a624bc80c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:08.825069 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cd3db-be07-4dd8-ab61-c2a624bc80c2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:08.825552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cd3db-be07-4dd8-ab61-c2a624bc80c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:08.826051 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22cd3db-be07-4dd8-ab61-c2a624bc80c2 None admin] PATH_INFO: `/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:08.827487 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22cd3db-be07-4dd8-ab61-c2a624bc80c2 None admin] RBAC: Authorizing `identity:get_group(group_id=a9d83e0a3baf472e90de2b6ec1a6f6f4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:08.829204 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22cd3db-be07-4dd8-ab61-c2a624bc80c2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:08.831481 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 204/399] 10.222.0.85 () {56 vars in 1180 bytes} [Wed May 25 15:54:08 2022] GET /identity/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4 => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:08.837040 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38594853-81e4-4bfc-8644-e3cb4aa902a6 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:08.959942 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38594853-81e4-4bfc-8644-e3cb4aa902a6 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:08.982979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:08.999537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68cdc364-2ed4-442c-a4c1-961796afef91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:09.000834 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdc364-2ed4-442c-a4c1-961796afef91 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:09.001337 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdc364-2ed4-442c-a4c1-961796afef91 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:09.001814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68cdc364-2ed4-442c-a4c1-961796afef91 None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:09.003345 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdc364-2ed4-442c-a4c1-961796afef91 None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:09.005198 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68cdc364-2ed4-442c-a4c1-961796afef91 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:09.007655 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 196/400] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:54:08 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:09.017086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cd3db-be07-4dd8-ab61-c2a624bc80c2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:09.111271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e178c-f468-4a73-81b2-934fcb3f8620 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:09.111962 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e178c-f468-4a73-81b2-934fcb3f8620 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:09.112442 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e178c-f468-4a73-81b2-934fcb3f8620 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:09.114861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 197/401] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:54:09.125321 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f60923-ad21-40d0-a81c-6606eb488c32 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:09.125885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f60923-ad21-40d0-a81c-6606eb488c32 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:09.126387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76f60923-ad21-40d0-a81c-6606eb488c32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:09.152814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-76f60923-ad21-40d0-a81c-6606eb488c32 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:09.155101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-76f60923-ad21-40d0-a81c-6606eb488c32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:09.156170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:09.159514 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22cd3db-be07-4dd8-ab61-c2a624bc80c2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:09.221868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e03404-b5e5-4548-825c-814d71649f4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:09.223118 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e03404-b5e5-4548-825c-814d71649f4c None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:09.224278 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e03404-b5e5-4548-825c-814d71649f4c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:09.224576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e03404-b5e5-4548-825c-814d71649f4c None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:09.226718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e03404-b5e5-4548-825c-814d71649f4c None admin] RBAC: Authorizing `identity:create_grant(project_id=a8be03eb22fb4d4d9fb156c025f259d7, group_id=a9d83e0a3baf472e90de2b6ec1a6f6f4, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:09.228219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e03404-b5e5-4548-825c-814d71649f4c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:09.254536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 205/402] 10.222.0.85 () {58 vars in 1442 bytes} [Wed May 25 15:54:09 2022] PUT /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:54:09.256149 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbf338-afaa-4e64-866f-3ca34ae53afd None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:09.263848 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbf338-afaa-4e64-866f-3ca34ae53afd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:09.264709 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0bbf338-afaa-4e64-866f-3ca34ae53afd None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:09.266723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 206/403] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:09 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:09.271614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 198/404] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:09.273261 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ec2b7-b604-4a20-98f1-a7d18c73f675 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:09.273752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ec2b7-b604-4a20-98f1-a7d18c73f675 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:09.274224 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b88ec2b7-b604-4a20-98f1-a7d18c73f675 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:09.293710 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f60923-ad21-40d0-a81c-6606eb488c32 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:09.325012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b88ec2b7-b604-4a20-98f1-a7d18c73f675 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:09.326610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b88ec2b7-b604-4a20-98f1-a7d18c73f675 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:09.410421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76f60923-ad21-40d0-a81c-6606eb488c32 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:09.431282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:09.458560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 207/405] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:09.463726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1447d9-509d-4daa-bd02-fc580335dfb6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:09.464620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1447d9-509d-4daa-bd02-fc580335dfb6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:09.464916 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1447d9-509d-4daa-bd02-fc580335dfb6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:09.465200 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb1447d9-509d-4daa-bd02-fc580335dfb6 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:09.478088 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb1447d9-509d-4daa-bd02-fc580335dfb6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:09.486966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-fb1447d9-509d-4daa-bd02-fc580335dfb6 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:54:09.486966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:09.486966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:09.486966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:09.486966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:09.486966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:09.486966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:09.486966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:09.486966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:09.486966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:09.486966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:09.486966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:09.486966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:09.486966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:09.486966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:09.486966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:09.488500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:09.488500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:09.488500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:09.488500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:09.488500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:09.488500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:09.488500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:09.488500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:09.488500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:09.488500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:09.488500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:09.488500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:09.488500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:09.488500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:09.488500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:09.488500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:09.489313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:09.489313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:09.489313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:09.489313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:09.489313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:09.489313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:09.489313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:09.489313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:09.489313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:09.489313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:09.489313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:54:09.489313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:09.493640 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 199/406] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:54:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:09.495084 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ec2b7-b604-4a20-98f1-a7d18c73f675 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:09.504359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1447d9-509d-4daa-bd02-fc580335dfb6 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:09.625309 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb1447d9-509d-4daa-bd02-fc580335dfb6 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:09.631978 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:09.656608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b88ec2b7-b604-4a20-98f1-a7d18c73f675 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:09.680909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f38a33-ea55-4a83-af6d-ed7600491668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:09.681601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f38a33-ea55-4a83-af6d-ed7600491668 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:09.681893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f38a33-ea55-4a83-af6d-ed7600491668 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:09.682178 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f38a33-ea55-4a83-af6d-ed7600491668 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:09.682946 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f38a33-ea55-4a83-af6d-ed7600491668 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:09.683265 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f38a33-ea55-4a83-af6d-ed7600491668 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:09.684788 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f38a33-ea55-4a83-af6d-ed7600491668 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:09.693718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 200/407] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:09.698864 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f38a33-ea55-4a83-af6d-ed7600491668 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:09.718521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7926c0-c01f-42e4-ac49-f17561c696f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:09.719389 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7926c0-c01f-42e4-ac49-f17561c696f6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:09.720175 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7926c0-c01f-42e4-ac49-f17561c696f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:09.720464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7926c0-c01f-42e4-ac49-f17561c696f6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:09.741793 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7926c0-c01f-42e4-ac49-f17561c696f6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:09.755517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ee7926c0-c01f-42e4-ac49-f17561c696f6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:09.755517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:09.755517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:09.755517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:09.755517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:09.755517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:09.755517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:09.755517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:09.755517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:09.755517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:09.755517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:09.755517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:09.755517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:09.755517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:09.755517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:09.755517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:09.756441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:09.756441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:09.756441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:09.756441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:09.756441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:09.756441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:09.756441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:09.756441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:09.756441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:09.756441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:09.756441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:09.756441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:09.756441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:09.756441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:09.756441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:09.756441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:09.757244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:09.757244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:09.757244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:09.757244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:09.757244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:09.757244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:09.757244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:09.757244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:09.757244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:09.757244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:09.757244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:09.757244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:09.757244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 208/408] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:54:09 2022] GET /identity/v3/roles/admin => generated 83 bytes in 262 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:09.763910 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7926c0-c01f-42e4-ac49-f17561c696f6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:09.766625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:09.867796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f38a33-ea55-4a83-af6d-ed7600491668 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:09.930518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d6013-e6dc-4465-90c8-d06f21e47bef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:09.931406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d6013-e6dc-4465-90c8-d06f21e47bef None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:09.931697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d6013-e6dc-4465-90c8-d06f21e47bef None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:09.932006 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154d6013-e6dc-4465-90c8-d06f21e47bef None admin] PATH_INFO: `/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:09.933384 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d6013-e6dc-4465-90c8-d06f21e47bef None admin] RBAC: Authorizing `identity:get_group(group_id=a9d83e0a3baf472e90de2b6ec1a6f6f4)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:09.934850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-154d6013-e6dc-4465-90c8-d06f21e47bef None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:09.939227 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 201/409] 10.222.0.85 () {56 vars in 1180 bytes} [Wed May 25 15:54:09 2022] GET /identity/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4 => generated 226 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:09.945032 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d6013-e6dc-4465-90c8-d06f21e47bef None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:09.956642 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7926c0-c01f-42e4-ac49-f17561c696f6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:10.028999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433ddf8-c688-495b-b960-3610b98ca529 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:10.028999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433ddf8-c688-495b-b960-3610b98ca529 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:10.028999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433ddf8-c688-495b-b960-3610b98ca529 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:10.028999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433ddf8-c688-495b-b960-3610b98ca529 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:10.029388 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433ddf8-c688-495b-b960-3610b98ca529 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:10.029388 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433ddf8-c688-495b-b960-3610b98ca529 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:10.029789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433ddf8-c688-495b-b960-3610b98ca529 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:10.055210 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 209/410] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:10.060928 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433ddf8-c688-495b-b960-3610b98ca529 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:10.075762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:10.164258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154d6013-e6dc-4465-90c8-d06f21e47bef None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:10.174362 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:10.206800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9bede-6061-4ffc-b8a2-72d0eb949f07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:10.207690 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9bede-6061-4ffc-b8a2-72d0eb949f07 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:10.208004 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9bede-6061-4ffc-b8a2-72d0eb949f07 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:10.210240 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12e9bede-6061-4ffc-b8a2-72d0eb949f07 None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:10.211680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9bede-6061-4ffc-b8a2-72d0eb949f07 None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:10.213187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12e9bede-6061-4ffc-b8a2-72d0eb949f07 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:10.214896 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 202/411] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:54:09 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 273 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:10.221188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9bede-6061-4ffc-b8a2-72d0eb949f07 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:10.231583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433ddf8-c688-495b-b960-3610b98ca529 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:10.318405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9197c4-ff16-4fb8-812e-19d099803324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:10.322642 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9197c4-ff16-4fb8-812e-19d099803324 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:10.323074 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9197c4-ff16-4fb8-812e-19d099803324 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:10.323603 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac9197c4-ff16-4fb8-812e-19d099803324 None admin] PATH_INFO: `/v3/groups/6b5136a4c5994d6d824c7324ea0487a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:10.326293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9197c4-ff16-4fb8-812e-19d099803324 None admin] RBAC: Authorizing `identity:get_group(group_id=6b5136a4c5994d6d824c7324ea0487a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:10.329102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac9197c4-ff16-4fb8-812e-19d099803324 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:10.335582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 210/412] 10.222.0.85 () {56 vars in 1180 bytes} [Wed May 25 15:54:10 2022] GET /identity/v3/groups/6b5136a4c5994d6d824c7324ea0487a1 => generated 229 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:10.340988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9197c4-ff16-4fb8-812e-19d099803324 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:10.398403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12e9bede-6061-4ffc-b8a2-72d0eb949f07 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:10.439598 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02a3ca5-8fb2-4d2d-8d0d-62e5cb3936cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:10.441043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a3ca5-8fb2-4d2d-8d0d-62e5cb3936cc None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:10.442175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a3ca5-8fb2-4d2d-8d0d-62e5cb3936cc None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:10.442884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02a3ca5-8fb2-4d2d-8d0d-62e5cb3936cc None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4/roles/1cc9b5d798c648c986537d006320581f` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:10.447106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a3ca5-8fb2-4d2d-8d0d-62e5cb3936cc None admin] RBAC: Authorizing `identity:create_grant(project_id=a8be03eb22fb4d4d9fb156c025f259d7, group_id=a9d83e0a3baf472e90de2b6ec1a6f6f4, role_id=1cc9b5d798c648c986537d006320581f)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:10.449139 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e02a3ca5-8fb2-4d2d-8d0d-62e5cb3936cc None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:10.469945 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 203/413] 10.222.0.85 () {58 vars in 1442 bytes} [Wed May 25 15:54:10 2022] PUT /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4/roles/1cc9b5d798c648c986537d006320581f => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:54:10.471906 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44202576-ac79-47e8-8301-01f70b7e03cb None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:10.472208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44202576-ac79-47e8-8301-01f70b7e03cb None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:10.472492 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44202576-ac79-47e8-8301-01f70b7e03cb None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:10.475133 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 204/414] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:10.481114 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f2c8c-a500-455d-a638-8847e8f08be0 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:10.481440 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f2c8c-a500-455d-a638-8847e8f08be0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:10.481688 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485f2c8c-a500-455d-a638-8847e8f08be0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:10.489576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:10.514984 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac9197c4-ff16-4fb8-812e-19d099803324 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:10.525007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-485f2c8c-a500-455d-a638-8847e8f08be0 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:10.526214 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-485f2c8c-a500-455d-a638-8847e8f08be0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:10.560310 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce81f2-58d7-47c1-a3a3-2c8cda4803d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:10.563072 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce81f2-58d7-47c1-a3a3-2c8cda4803d1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:10.564761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce81f2-58d7-47c1-a3a3-2c8cda4803d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:10.565267 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ce81f2-58d7-47c1-a3a3-2c8cda4803d1 None admin] PATH_INFO: `/v3/projects/12da861032034c60b092c703b1fe5e90` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:10.569155 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce81f2-58d7-47c1-a3a3-2c8cda4803d1 None admin] RBAC: Authorizing `identity:get_project(project_id=12da861032034c60b092c703b1fe5e90)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:10.573448 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ce81f2-58d7-47c1-a3a3-2c8cda4803d1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:10.577502 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 211/415] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:54:10 2022] GET /identity/v3/projects/12da861032034c60b092c703b1fe5e90 => generated 344 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:10.583134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce81f2-58d7-47c1-a3a3-2c8cda4803d1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:10.592188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:10.663856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 205/416] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:10.665623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65b31e-1563-4e9f-bf4f-b75896bf40b5 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:10.665918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65b31e-1563-4e9f-bf4f-b75896bf40b5 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:10.666201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e65b31e-1563-4e9f-bf4f-b75896bf40b5 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:10.668673 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 206/417] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:10.674311 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff4295-a12c-46ef-8092-99306a95b180 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:10.674600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff4295-a12c-46ef-8092-99306a95b180 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:10.674882 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff4295-a12c-46ef-8092-99306a95b180 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:10.693190 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-95ff4295-a12c-46ef-8092-99306a95b180 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:10.694410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-95ff4295-a12c-46ef-8092-99306a95b180 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:10.760253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ce81f2-58d7-47c1-a3a3-2c8cda4803d1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:10.787969 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 207/418] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:54:10.789533 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff4295-a12c-46ef-8092-99306a95b180 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:10.796185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e637a5-55e6-46a6-825d-8450061c1adb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:10.797141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e637a5-55e6-46a6-825d-8450061c1adb None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:10.797436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e637a5-55e6-46a6-825d-8450061c1adb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:10.797720 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e637a5-55e6-46a6-825d-8450061c1adb None admin] PATH_INFO: `/v3/projects/12da861032034c60b092c703b1fe5e90/groups/6b5136a4c5994d6d824c7324ea0487a1/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:10.799846 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e637a5-55e6-46a6-825d-8450061c1adb None admin] RBAC: Authorizing `identity:create_grant(project_id=12da861032034c60b092c703b1fe5e90, group_id=6b5136a4c5994d6d824c7324ea0487a1, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:10.801438 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e637a5-55e6-46a6-825d-8450061c1adb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:10.829101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 212/419] 10.222.0.85 () {58 vars in 1442 bytes} [Wed May 25 15:54:10 2022] PUT /identity/v3/projects/12da861032034c60b092c703b1fe5e90/groups/6b5136a4c5994d6d824c7324ea0487a1/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:54:10.830759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e637a5-55e6-46a6-825d-8450061c1adb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:10.836331 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:10.952251 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:10.957331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff4295-a12c-46ef-8092-99306a95b180 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:11.013058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efaca37-29a7-41c3-b6ee-766ca5c88be3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:11.014374 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaca37-29a7-41c3-b6ee-766ca5c88be3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:11.016162 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaca37-29a7-41c3-b6ee-766ca5c88be3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:11.017499 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efaca37-29a7-41c3-b6ee-766ca5c88be3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:11.042538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efaca37-29a7-41c3-b6ee-766ca5c88be3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:11.062075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-2efaca37-29a7-41c3-b6ee-766ca5c88be3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:54:11.062075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:11.062075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:11.062075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:11.062075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:11.062075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:11.062075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:11.062075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:11.062075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:11.062075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:11.062075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:11.062075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:11.062075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:11.062075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:11.062075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:11.062075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:11.063060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:11.063060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:11.063060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:11.063060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:11.063060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:11.063060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:11.063060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:11.063060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:11.063060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:11.063060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:11.063060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:11.063060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:11.063060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:11.063060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:11.063060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:11.063060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:11.064539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:11.064539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:11.064539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:11.064539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:11.064539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:11.064539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:11.064539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:11.064539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:11.064539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:11.064539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:11.064539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:54:11.064539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:11.065165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e637a5-55e6-46a6-825d-8450061c1adb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:11.070399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 208/420] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:11.077176 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efaca37-29a7-41c3-b6ee-766ca5c88be3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:11.114140 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a327fb3-789e-47cd-8a65-b0938dfc0097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:11.115429 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a327fb3-789e-47cd-8a65-b0938dfc0097 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:11.115885 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a327fb3-789e-47cd-8a65-b0938dfc0097 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:11.116334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a327fb3-789e-47cd-8a65-b0938dfc0097 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:11.130162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a327fb3-789e-47cd-8a65-b0938dfc0097 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:11.140342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9a327fb3-789e-47cd-8a65-b0938dfc0097 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:54:11.140342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:11.140342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:11.140342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:11.140342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:11.140342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:11.140342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:11.140342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:11.140342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:11.140342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:11.140342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:11.140342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:11.140342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:11.140342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:11.140342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:11.140342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:11.141452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:11.141452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:11.141452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:11.141452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:11.141452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:11.141452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:11.141452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:11.141452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:11.141452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:11.141452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:11.141452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:11.141452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:11.141452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:11.141452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:11.141452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:11.141452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:11.142466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:11.142466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:11.142466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:11.142466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:11.142466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:11.142466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:11.142466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:11.142466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:11.142466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:11.142466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:11.142466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:54:11.142466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:11.149989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 213/421] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:54:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:11.151744 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a327fb3-789e-47cd-8a65-b0938dfc0097 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:11.196976 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efaca37-29a7-41c3-b6ee-766ca5c88be3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:11.242679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37588cbc-a2e7-498e-be1f-02e9d523e8ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:11.243367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37588cbc-a2e7-498e-be1f-02e9d523e8ce None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:11.243655 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37588cbc-a2e7-498e-be1f-02e9d523e8ce None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:11.243968 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37588cbc-a2e7-498e-be1f-02e9d523e8ce None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:11.244730 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37588cbc-a2e7-498e-be1f-02e9d523e8ce None admin] RBAC: Adding query filter params (name=member) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:11.245044 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37588cbc-a2e7-498e-be1f-02e9d523e8ce None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:11.250848 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37588cbc-a2e7-498e-be1f-02e9d523e8ce None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:11.260159 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:11.271064 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 209/422] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:54:11 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:11.293488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37588cbc-a2e7-498e-be1f-02e9d523e8ce None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:11.323099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:11.324458 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a327fb3-789e-47cd-8a65-b0938dfc0097 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:11.396592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0d285-754f-47d4-9f37-c2d87c9ace54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:11.397326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0d285-754f-47d4-9f37-c2d87c9ace54 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:11.397617 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0d285-754f-47d4-9f37-c2d87c9ace54 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:11.397894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b0d285-754f-47d4-9f37-c2d87c9ace54 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:11.398652 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0d285-754f-47d4-9f37-c2d87c9ace54 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:11.398964 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0d285-754f-47d4-9f37-c2d87c9ace54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:11.403079 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b0d285-754f-47d4-9f37-c2d87c9ace54 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:11.419726 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 214/423] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:11.424665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0d285-754f-47d4-9f37-c2d87c9ace54 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:11.447762 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37588cbc-a2e7-498e-be1f-02e9d523e8ce None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:11.527120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9902023-af34-4267-8c7b-1967fd6d52b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:11.527785 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9902023-af34-4267-8c7b-1967fd6d52b0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:11.528099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9902023-af34-4267-8c7b-1967fd6d52b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:11.528380 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9902023-af34-4267-8c7b-1967fd6d52b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:11.564945 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b0d285-754f-47d4-9f37-c2d87c9ace54 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:11.593868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:11.605023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec77e5-9f17-40e8-b68f-06f13b0a583a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:11.606884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec77e5-9f17-40e8-b68f-06f13b0a583a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:11.608262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec77e5-9f17-40e8-b68f-06f13b0a583a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:11.608555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec77e5-9f17-40e8-b68f-06f13b0a583a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:11.663779 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9902023-af34-4267-8c7b-1967fd6d52b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:11.665888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9902023-af34-4267-8c7b-1967fd6d52b0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:11.704184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:11.722826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec77e5-9f17-40e8-b68f-06f13b0a583a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:11.724363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acec77e5-9f17-40e8-b68f-06f13b0a583a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:11.846035 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 215/424] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 15:54:11.848109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec77e5-9f17-40e8-b68f-06f13b0a583a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:11.896250 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:11.963269 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 210/425] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 670 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 15:54:11.973092 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec77e5-9f17-40e8-b68f-06f13b0a583a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:11.980861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9902023-af34-4267-8c7b-1967fd6d52b0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:12.012318 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded2c40d-8fe8-45cd-ab51-5d24b580d56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:12.013503 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2c40d-8fe8-45cd-ab51-5d24b580d56c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:12.013795 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2c40d-8fe8-45cd-ab51-5d24b580d56c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:12.014076 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded2c40d-8fe8-45cd-ab51-5d24b580d56c None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:12.015590 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2c40d-8fe8-45cd-ab51-5d24b580d56c None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:12.017688 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ded2c40d-8fe8-45cd-ab51-5d24b580d56c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:12.020195 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 216/426] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:54:11 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:12.021646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9db48-e355-49c8-bc58-3b6428ef1102 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:12.023311 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9db48-e355-49c8-bc58-3b6428ef1102 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:12.023311 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e9db48-e355-49c8-bc58-3b6428ef1102 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:12.027812 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 217/427] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:12.027812 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9db48-e355-49c8-bc58-3b6428ef1102 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:12.096599 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9902023-af34-4267-8c7b-1967fd6d52b0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:12.114544 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:12.133764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0b24e6-980a-4643-82ad-4342b6999c3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:12.135101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b24e6-980a-4643-82ad-4342b6999c3e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:12.135616 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b24e6-980a-4643-82ad-4342b6999c3e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:12.136126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0b24e6-980a-4643-82ad-4342b6999c3e None admin] PATH_INFO: `/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:12.137715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b24e6-980a-4643-82ad-4342b6999c3e None admin] RBAC: Authorizing `identity:get_project(project_id=89a647c5dfc34a43ad12c49f8f57d99d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:12.139679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0b24e6-980a-4643-82ad-4342b6999c3e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:12.141830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 211/428] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:54:11 2022] GET /identity/v3/projects/89a647c5dfc34a43ad12c49f8f57d99d => generated 298 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:12.143394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b04d33-75f9-4f3f-bc06-df1a7c261250 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:12.143933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b04d33-75f9-4f3f-bc06-df1a7c261250 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:12.144227 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:12.144663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96b04d33-75f9-4f3f-bc06-df1a7c261250 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:12.150007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e9db48-e355-49c8-bc58-3b6428ef1102 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:12.167080 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-96b04d33-75f9-4f3f-bc06-df1a7c261250 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:12.168566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-96b04d33-75f9-4f3f-bc06-df1a7c261250 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:12.202950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86ffdc-7688-4b4c-814f-7867ce28b094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:12.203850 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86ffdc-7688-4b4c-814f-7867ce28b094 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:12.204153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86ffdc-7688-4b4c-814f-7867ce28b094 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:12.204429 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd86ffdc-7688-4b4c-814f-7867ce28b094 None admin] PATH_INFO: `/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:12.206205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86ffdc-7688-4b4c-814f-7867ce28b094 None admin] RBAC: Authorizing `identity:get_group(group_id=a9d83e0a3baf472e90de2b6ec1a6f6f4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:12.207665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd86ffdc-7688-4b4c-814f-7867ce28b094 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:12.214505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 218/429] 10.222.0.85 () {56 vars in 1180 bytes} [Wed May 25 15:54:12 2022] GET /identity/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4 => generated 226 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:12.214505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86ffdc-7688-4b4c-814f-7867ce28b094 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:12.254053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 212/430] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:12.255330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b04d33-75f9-4f3f-bc06-df1a7c261250 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:12.324202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd86ffdc-7688-4b4c-814f-7867ce28b094 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:12.359778 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f05d1-f8b5-4911-8062-07094dad2833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:12.360845 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f05d1-f8b5-4911-8062-07094dad2833 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:12.361238 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f05d1-f8b5-4911-8062-07094dad2833 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:12.361592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f05d1-f8b5-4911-8062-07094dad2833 None admin] PATH_INFO: `/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:12.362886 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f05d1-f8b5-4911-8062-07094dad2833 None admin] RBAC: Authorizing `identity:get_group(group_id=a9d83e0a3baf472e90de2b6ec1a6f6f4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:12.364469 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f05d1-f8b5-4911-8062-07094dad2833 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:12.366547 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 219/431] 10.222.0.85 () {56 vars in 1180 bytes} [Wed May 25 15:54:12 2022] GET /identity/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4 => generated 226 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:12.372062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f05d1-f8b5-4911-8062-07094dad2833 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:12.374005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96b04d33-75f9-4f3f-bc06-df1a7c261250 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:12.391423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:12.403549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:12.410083 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1fcda-fb71-46de-8a54-52fcdc99e49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:12.410942 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1fcda-fb71-46de-8a54-52fcdc99e49a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:12.411371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1fcda-fb71-46de-8a54-52fcdc99e49a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:12.411848 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bc1fcda-fb71-46de-8a54-52fcdc99e49a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:12.412829 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc1fcda-fb71-46de-8a54-52fcdc99e49a None admin] RBAC: Adding query filter params (group.id=a9d83e0a3baf472e90de2b6ec1a6f6f4, role.id=1cc9b5d798c648c986537d006320581f, scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:12.413288 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc1fcda-fb71-46de-8a54-52fcdc99e49a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:12.414893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bc1fcda-fb71-46de-8a54-52fcdc99e49a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:12.427955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 213/432] 10.222.0.85 () {56 vars in 1376 bytes} [Wed May 25 15:54:12 2022] GET /identity/v3/role_assignments?group.id=a9d83e0a3baf472e90de2b6ec1a6f6f4&scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d&role.id=1cc9b5d798c648c986537d006320581f => generated 611 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:12.429231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1fcda-fb71-46de-8a54-52fcdc99e49a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:12.485583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f05d1-f8b5-4911-8062-07094dad2833 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:12.528027 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4897d7-7c38-4aff-bc6f-91afbb8fc73d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:12.528854 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4897d7-7c38-4aff-bc6f-91afbb8fc73d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:12.529197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4897d7-7c38-4aff-bc6f-91afbb8fc73d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:12.529474 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d4897d7-7c38-4aff-bc6f-91afbb8fc73d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:12.553028 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d4897d7-7c38-4aff-bc6f-91afbb8fc73d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:12.566823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-5d4897d7-7c38-4aff-bc6f-91afbb8fc73d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. May 25 15:54:12.566823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:12.566823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:12.566823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:12.566823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:12.566823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:12.566823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:12.566823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:12.566823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:12.566823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:12.566823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:12.566823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:12.566823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:12.566823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:12.566823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:12.566823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:12.568269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:12.568269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:12.568269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:12.568269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:12.568269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:12.568269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:12.568269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:12.568269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:12.568269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:12.568269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:12.568269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:12.568269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:12.568269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:12.568269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:12.568269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:12.568269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:12.569288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:12.569288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:12.569288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:12.569288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:12.569288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:12.569288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:12.569288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:12.569288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:12.569288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:12.569288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:12.569288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. May 25 15:54:12.569288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:12.569288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 220/433] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:12.574268 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4897d7-7c38-4aff-bc6f-91afbb8fc73d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:12.669887 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bc1fcda-fb71-46de-8a54-52fcdc99e49a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:12.703105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:12.716196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d4897d7-7c38-4aff-bc6f-91afbb8fc73d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:12.738196 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42583880-0cd9-4f72-8036-493fd20b7f56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:12.738791 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42583880-0cd9-4f72-8036-493fd20b7f56 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:12.739078 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42583880-0cd9-4f72-8036-493fd20b7f56 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:12.739373 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42583880-0cd9-4f72-8036-493fd20b7f56 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:12.740241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42583880-0cd9-4f72-8036-493fd20b7f56 None admin] RBAC: Adding query filter params (group.id=a9d83e0a3baf472e90de2b6ec1a6f6f4, role.id=9c81542451e54c0ea22a20c026be6be7, scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:12.740564 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42583880-0cd9-4f72-8036-493fd20b7f56 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:12.743727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42583880-0cd9-4f72-8036-493fd20b7f56 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:12.754542 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 214/434] 10.222.0.85 () {56 vars in 1376 bytes} [Wed May 25 15:54:12 2022] GET /identity/v3/role_assignments?group.id=a9d83e0a3baf472e90de2b6ec1a6f6f4&scope.project.id=89a647c5dfc34a43ad12c49f8f57d99d&role.id=9c81542451e54c0ea22a20c026be6be7 => generated 611 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:12.762928 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5cd025-1cbf-40bb-b765-c5c3149705bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:12.764085 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cd025-1cbf-40bb-b765-c5c3149705bd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:12.764708 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cd025-1cbf-40bb-b765-c5c3149705bd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:12.765216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5cd025-1cbf-40bb-b765-c5c3149705bd None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:12.766257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5cd025-1cbf-40bb-b765-c5c3149705bd None admin] RBAC: Adding query filter params (name=member) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:12.766798 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5cd025-1cbf-40bb-b765-c5c3149705bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:12.768588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5cd025-1cbf-40bb-b765-c5c3149705bd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:12.781922 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 221/435] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:54:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:12.787272 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42583880-0cd9-4f72-8036-493fd20b7f56 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:12.842092 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:12.902956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42583880-0cd9-4f72-8036-493fd20b7f56 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:12.940707 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d933fd-4225-45a8-bd9c-f92051c22467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:12.941767 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d933fd-4225-45a8-bd9c-f92051c22467 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:12.942281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d933fd-4225-45a8-bd9c-f92051c22467 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:12.942756 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21d933fd-4225-45a8-bd9c-f92051c22467 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:12.977290 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e68b1-68b6-42f3-a09c-432fb0bea8da None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:12.978082 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e68b1-68b6-42f3-a09c-432fb0bea8da None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:12.978582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7e68b1-68b6-42f3-a09c-432fb0bea8da None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:12.981749 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 222/436] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:12.986270 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d69769-a221-4eaa-a66f-9599f5cec166 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:12.986566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d69769-a221-4eaa-a66f-9599f5cec166 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:12.986854 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d69769-a221-4eaa-a66f-9599f5cec166 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:13.008384 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-31d69769-a221-4eaa-a66f-9599f5cec166 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:13.009523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-31d69769-a221-4eaa-a66f-9599f5cec166 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:13.023717 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d933fd-4225-45a8-bd9c-f92051c22467 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:13.025462 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21d933fd-4225-45a8-bd9c-f92051c22467 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:13.077905 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:13.105293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 223/437] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:12 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:13.129573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d69769-a221-4eaa-a66f-9599f5cec166 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:13.156025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 215/438] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 369 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 15:54:13.162204 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d933fd-4225-45a8-bd9c-f92051c22467 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:13.242173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:13.269263 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d69769-a221-4eaa-a66f-9599f5cec166 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:13.282700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21d933fd-4225-45a8-bd9c-f92051c22467 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:13.319486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575a9dc1-a131-4092-b1f3-1affb070c929 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:13.320520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a9dc1-a131-4092-b1f3-1affb070c929 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:13.320813 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a9dc1-a131-4092-b1f3-1affb070c929 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:13.321959 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575a9dc1-a131-4092-b1f3-1affb070c929 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:13.328551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0980dbe2-c441-4af0-8df8-492bdda7a6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:13.329441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980dbe2-c441-4af0-8df8-492bdda7a6f6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:13.329724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980dbe2-c441-4af0-8df8-492bdda7a6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:13.330006 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0980dbe2-c441-4af0-8df8-492bdda7a6f6 None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:13.331202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980dbe2-c441-4af0-8df8-492bdda7a6f6 None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:13.333145 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0980dbe2-c441-4af0-8df8-492bdda7a6f6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:13.333272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575a9dc1-a131-4092-b1f3-1affb070c929 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:13.335043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 216/439] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:54:13 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:13.336262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5365625-097e-4f49-95d6-f2393ec4ff2d None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:13.336554 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5365625-097e-4f49-95d6-f2393ec4ff2d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:13.336839 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5365625-097e-4f49-95d6-f2393ec4ff2d None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:13.338050 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 217/440] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:13.341013 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5365625-097e-4f49-95d6-f2393ec4ff2d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:13.341337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-575a9dc1-a131-4092-b1f3-1affb070c929 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:54:13.341337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:13.341337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:13.341337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:13.341337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:13.341337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:13.341337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:13.341337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:13.341337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:13.341337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:13.341337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:13.341337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:13.341337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:13.341337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:13.341337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:13.341337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:13.342266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:13.342266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:13.342266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:13.342266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:13.342266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:13.342266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:13.342266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:13.342266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:13.342266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:13.342266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:13.342266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:13.342266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:13.342266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:13.342266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:13.342266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:13.342266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:13.343462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:13.343462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:13.343462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:13.343462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:13.343462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:13.343462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:13.343462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:13.343462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:13.343462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:13.343462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:13.343462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:54:13.343462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:13.343462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 224/441] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:54:13 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:13.344217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb08f33-5d8d-492b-9e0f-88667758a379 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:13.344469 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb08f33-5d8d-492b-9e0f-88667758a379 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:13.344957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb08f33-5d8d-492b-9e0f-88667758a379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:13.362083 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:13.362265 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1cb08f33-5d8d-492b-9e0f-88667758a379 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:13.362465 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1cb08f33-5d8d-492b-9e0f-88667758a379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:13.433943 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 225/442] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:13.434600 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb08f33-5d8d-492b-9e0f-88667758a379 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:13.461033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5365625-097e-4f49-95d6-f2393ec4ff2d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:13.497340 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30ef28-89d4-4595-a12d-23cec67f1daa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:13.498220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30ef28-89d4-4595-a12d-23cec67f1daa None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:13.498535 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30ef28-89d4-4595-a12d-23cec67f1daa None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:13.498822 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c30ef28-89d4-4595-a12d-23cec67f1daa None admin] PATH_INFO: `/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:13.500017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30ef28-89d4-4595-a12d-23cec67f1daa None admin] RBAC: Authorizing `identity:get_group(group_id=a9d83e0a3baf472e90de2b6ec1a6f6f4)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:13.501481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c30ef28-89d4-4595-a12d-23cec67f1daa None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:13.503157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 218/443] 10.222.0.85 () {56 vars in 1180 bytes} [Wed May 25 15:54:13 2022] GET /identity/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4 => generated 226 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:13.503952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30ef28-89d4-4595-a12d-23cec67f1daa None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:13.512935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:13.542100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb08f33-5d8d-492b-9e0f-88667758a379 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:13.576822 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051dc100-5071-44d8-961f-d9ebb49065fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:13.577500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dc100-5071-44d8-961f-d9ebb49065fa None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:13.577776 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dc100-5071-44d8-961f-d9ebb49065fa None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:13.578038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-051dc100-5071-44d8-961f-d9ebb49065fa None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:13.578795 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051dc100-5071-44d8-961f-d9ebb49065fa None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:13.579107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051dc100-5071-44d8-961f-d9ebb49065fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:13.580531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-051dc100-5071-44d8-961f-d9ebb49065fa None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:13.580756 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:13.589620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 226/444] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:13 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:13.590413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051dc100-5071-44d8-961f-d9ebb49065fa None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:13.601252 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c30ef28-89d4-4595-a12d-23cec67f1daa None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:13.635270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38460483-6fc6-4428-a410-3a997773cd9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:13.636116 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38460483-6fc6-4428-a410-3a997773cd9e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:13.636399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38460483-6fc6-4428-a410-3a997773cd9e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:13.636676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38460483-6fc6-4428-a410-3a997773cd9e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:13.646617 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38460483-6fc6-4428-a410-3a997773cd9e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:13.654333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-38460483-6fc6-4428-a410-3a997773cd9e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:13.654333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:13.654333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:13.654333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:13.654333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:13.654333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:13.654333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:13.654333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:13.654333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:13.654333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:13.654333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:13.654333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:13.654333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:13.654333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:13.654333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:13.654333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:13.655335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:13.655335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:13.655335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:13.655335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:13.655335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:13.655335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:13.655335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:13.655335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:13.655335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:13.655335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:13.655335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:13.655335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:13.655335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:13.655335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:13.655335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:13.655335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:13.656201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:13.656201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:13.656201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:13.656201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:13.656201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:13.656201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:13.656201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:13.656201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:13.656201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:13.656201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:13.656201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:13.656201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:13.656201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 219/445] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:54:13 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:13.657072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38460483-6fc6-4428-a410-3a997773cd9e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:13.696055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-051dc100-5071-44d8-961f-d9ebb49065fa None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:13.729653 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a0e20-8c3b-49e8-b5a6-5b083109a89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:13.730308 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a0e20-8c3b-49e8-b5a6-5b083109a89e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:13.730596 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a0e20-8c3b-49e8-b5a6-5b083109a89e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:13.730881 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-793a0e20-8c3b-49e8-b5a6-5b083109a89e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:13.731722 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a0e20-8c3b-49e8-b5a6-5b083109a89e None admin] RBAC: Adding query filter params (group.id=a9d83e0a3baf472e90de2b6ec1a6f6f4, role.id=9c81542451e54c0ea22a20c026be6be7, scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:13.732070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a0e20-8c3b-49e8-b5a6-5b083109a89e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:13.733550 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-793a0e20-8c3b-49e8-b5a6-5b083109a89e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:13.735443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:13.751966 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 227/446] 10.222.0.85 () {56 vars in 1376 bytes} [Wed May 25 15:54:13 2022] GET /identity/v3/role_assignments?group.id=a9d83e0a3baf472e90de2b6ec1a6f6f4&scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7&role.id=9c81542451e54c0ea22a20c026be6be7 => generated 611 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:13.751966 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a0e20-8c3b-49e8-b5a6-5b083109a89e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:13.753104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38460483-6fc6-4428-a410-3a997773cd9e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:13.796531 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191f0ca-0ac6-41ed-8365-24e70b97186f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:13.797290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191f0ca-0ac6-41ed-8365-24e70b97186f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:13.797572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191f0ca-0ac6-41ed-8365-24e70b97186f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:13.797855 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191f0ca-0ac6-41ed-8365-24e70b97186f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:13.813664 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:13.875377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-793a0e20-8c3b-49e8-b5a6-5b083109a89e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:13.879666 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8191f0ca-0ac6-41ed-8365-24e70b97186f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:13.881210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8191f0ca-0ac6-41ed-8365-24e70b97186f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:13.913752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818d940-71f3-4111-a322-2f9ebe1e49c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:13.914443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818d940-71f3-4111-a322-2f9ebe1e49c9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:13.914739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818d940-71f3-4111-a322-2f9ebe1e49c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:13.915026 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a818d940-71f3-4111-a322-2f9ebe1e49c9 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:13.915794 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818d940-71f3-4111-a322-2f9ebe1e49c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:13.916191 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818d940-71f3-4111-a322-2f9ebe1e49c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:13.918021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a818d940-71f3-4111-a322-2f9ebe1e49c9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:13.926658 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 228/447] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:13.932578 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818d940-71f3-4111-a322-2f9ebe1e49c9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:13.999992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:14.001427 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 220/448] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:54:14.006269 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191f0ca-0ac6-41ed-8365-24e70b97186f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:14.056167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:14.056556 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a818d940-71f3-4111-a322-2f9ebe1e49c9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:14.096833 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c75dd-6e31-4f4b-9967-63746130c6ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:14.097764 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c75dd-6e31-4f4b-9967-63746130c6ba None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:14.098057 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c75dd-6e31-4f4b-9967-63746130c6ba None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:14.099055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-609c75dd-6e31-4f4b-9967-63746130c6ba None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:14.120643 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191f0ca-0ac6-41ed-8365-24e70b97186f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:14.156169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda08c86-f3b6-464e-b22b-343a9a98061d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:14.157084 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda08c86-f3b6-464e-b22b-343a9a98061d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:14.157369 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda08c86-f3b6-464e-b22b-343a9a98061d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:14.157648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda08c86-f3b6-464e-b22b-343a9a98061d None admin] PATH_INFO: `/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:14.158830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda08c86-f3b6-464e-b22b-343a9a98061d None admin] RBAC: Authorizing `identity:get_project(project_id=a8be03eb22fb4d4d9fb156c025f259d7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:14.160357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda08c86-f3b6-464e-b22b-343a9a98061d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:14.162280 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 221/449] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:54:14 2022] GET /identity/v3/projects/a8be03eb22fb4d4d9fb156c025f259d7 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:14.167462 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda08c86-f3b6-464e-b22b-343a9a98061d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:14.174308 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c75dd-6e31-4f4b-9967-63746130c6ba None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:14.175779 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-609c75dd-6e31-4f4b-9967-63746130c6ba None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:14.232420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:14.265231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda08c86-f3b6-464e-b22b-343a9a98061d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:14.272860 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 229/450] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:54:14.274152 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:14.278984 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c75dd-6e31-4f4b-9967-63746130c6ba None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:14.299837 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8992dc9-032c-463d-8820-5bb32a26c4e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:14.300727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8992dc9-032c-463d-8820-5bb32a26c4e7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:14.301015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8992dc9-032c-463d-8820-5bb32a26c4e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:14.301295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8992dc9-032c-463d-8820-5bb32a26c4e7 None admin] PATH_INFO: `/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:14.302496 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8992dc9-032c-463d-8820-5bb32a26c4e7 None admin] RBAC: Authorizing `identity:get_group(group_id=a9d83e0a3baf472e90de2b6ec1a6f6f4)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:14.304009 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8992dc9-032c-463d-8820-5bb32a26c4e7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:14.305627 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 222/451] 10.222.0.85 () {56 vars in 1180 bytes} [Wed May 25 15:54:14 2022] GET /identity/v3/groups/a9d83e0a3baf472e90de2b6ec1a6f6f4 => generated 226 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:14.311403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8992dc9-032c-463d-8820-5bb32a26c4e7 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:14.380314 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-609c75dd-6e31-4f4b-9967-63746130c6ba None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:14.406596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8992dc9-032c-463d-8820-5bb32a26c4e7 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:14.413241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7cd31-6f32-4080-97df-45295b111804 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:14.414117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7cd31-6f32-4080-97df-45295b111804 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:14.414413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7cd31-6f32-4080-97df-45295b111804 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:14.414694 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0c7cd31-6f32-4080-97df-45295b111804 None admin] PATH_INFO: `/v3/projects/12da861032034c60b092c703b1fe5e90` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:14.415913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7cd31-6f32-4080-97df-45295b111804 None admin] RBAC: Authorizing `identity:get_project(project_id=12da861032034c60b092c703b1fe5e90)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:14.417379 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0c7cd31-6f32-4080-97df-45295b111804 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:14.419354 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 230/452] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:54:14 2022] GET /identity/v3/projects/12da861032034c60b092c703b1fe5e90 => generated 344 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:14.424604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7cd31-6f32-4080-97df-45295b111804 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:14.444013 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2652c7-e4ec-4738-a548-a659483db62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:14.444677 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2652c7-e4ec-4738-a548-a659483db62e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:14.444964 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2652c7-e4ec-4738-a548-a659483db62e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:14.445243 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2652c7-e4ec-4738-a548-a659483db62e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:14.446076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2652c7-e4ec-4738-a548-a659483db62e None admin] RBAC: Adding query filter params (group.id=a9d83e0a3baf472e90de2b6ec1a6f6f4, role.id=1cc9b5d798c648c986537d006320581f, scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:14.446394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2652c7-e4ec-4738-a548-a659483db62e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:14.447853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2652c7-e4ec-4738-a548-a659483db62e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:14.450536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:14.453957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 223/453] 10.222.0.85 () {56 vars in 1376 bytes} [Wed May 25 15:54:14 2022] GET /identity/v3/role_assignments?group.id=a9d83e0a3baf472e90de2b6ec1a6f6f4&scope.project.id=a8be03eb22fb4d4d9fb156c025f259d7&role.id=1cc9b5d798c648c986537d006320581f => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:14.538903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0c7cd31-6f32-4080-97df-45295b111804 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:14.576726 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ab6841c-c8b7-4c35-811f-0728dd275eb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:14.577510 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6841c-c8b7-4c35-811f-0728dd275eb9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:14.578073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6841c-c8b7-4c35-811f-0728dd275eb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:14.578073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ab6841c-c8b7-4c35-811f-0728dd275eb9 None admin] PATH_INFO: `/v3/groups/6b5136a4c5994d6d824c7324ea0487a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:14.579316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6841c-c8b7-4c35-811f-0728dd275eb9 None admin] RBAC: Authorizing `identity:get_group(group_id=6b5136a4c5994d6d824c7324ea0487a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:14.580831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ab6841c-c8b7-4c35-811f-0728dd275eb9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:14.582505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 231/454] 10.222.0.85 () {56 vars in 1180 bytes} [Wed May 25 15:54:14 2022] GET /identity/v3/groups/6b5136a4c5994d6d824c7324ea0487a1 => generated 229 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:14.589054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2652c7-e4ec-4738-a548-a659483db62e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:14.635140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:14.697333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2652c7-e4ec-4738-a548-a659483db62e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:14.733860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ff74af-606d-4d93-b7fb-59bdfb64a9cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:14.734534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff74af-606d-4d93-b7fb-59bdfb64a9cd None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:14.734826 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff74af-606d-4d93-b7fb-59bdfb64a9cd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:14.735108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ff74af-606d-4d93-b7fb-59bdfb64a9cd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:14.736005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff74af-606d-4d93-b7fb-59bdfb64a9cd None admin] RBAC: Adding query filter params (group.id=6b5136a4c5994d6d824c7324ea0487a1, role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=12da861032034c60b092c703b1fe5e90) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:14.736308 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff74af-606d-4d93-b7fb-59bdfb64a9cd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:14.737793 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ff74af-606d-4d93-b7fb-59bdfb64a9cd None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:14.744002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 224/455] 10.222.0.85 () {56 vars in 1376 bytes} [Wed May 25 15:54:14 2022] GET /identity/v3/role_assignments?group.id=6b5136a4c5994d6d824c7324ea0487a1&scope.project.id=12da861032034c60b092c703b1fe5e90&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 611 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:17.367752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477f0c1-99f2-4852-a0f4-25fa8af6b028 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:17.368819 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477f0c1-99f2-4852-a0f4-25fa8af6b028 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:17.369302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e477f0c1-99f2-4852-a0f4-25fa8af6b028 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:17.371479 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 232/456] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:17.377165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a6efb-1cb5-4ff3-9582-d2b63d48bb87 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:17.378111 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a6efb-1cb5-4ff3-9582-d2b63d48bb87 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:17.378594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32a6efb-1cb5-4ff3-9582-d2b63d48bb87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:17.405040 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e32a6efb-1cb5-4ff3-9582-d2b63d48bb87 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:17.406694 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e32a6efb-1cb5-4ff3-9582-d2b63d48bb87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:17.491091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 225/457] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:17.523548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477f0c1-99f2-4852-a0f4-25fa8af6b028 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:17.646371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1a118-2934-456e-85c5-ace308cc4f7e None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:17.647087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1a118-2934-456e-85c5-ace308cc4f7e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:17.647569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b1a118-2934-456e-85c5-ace308cc4f7e None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:17.649425 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 226/458] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:17.651959 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:17.655127 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe58c50-9409-4193-bc97-2cfccf6ebee1 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:17.656119 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe58c50-9409-4193-bc97-2cfccf6ebee1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:17.656603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe58c50-9409-4193-bc97-2cfccf6ebee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:17.658521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e477f0c1-99f2-4852-a0f4-25fa8af6b028 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:17.686386 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-cfe58c50-9409-4193-bc97-2cfccf6ebee1 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:17.689003 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-cfe58c50-9409-4193-bc97-2cfccf6ebee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:17.707590 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad00685d-95e8-4855-ad4c-b5fbcfba62d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:17.709122 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00685d-95e8-4855-ad4c-b5fbcfba62d6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:17.709586 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00685d-95e8-4855-ad4c-b5fbcfba62d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:17.710036 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad00685d-95e8-4855-ad4c-b5fbcfba62d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:17.720881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:17.722580 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00685d-95e8-4855-ad4c-b5fbcfba62d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:17.724401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad00685d-95e8-4855-ad4c-b5fbcfba62d6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:17.743683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-ad00685d-95e8-4855-ad4c-b5fbcfba62d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:17.743683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:17.743683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:17.743683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:17.743683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:17.743683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:17.743683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:17.743683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:17.743683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:17.743683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:17.743683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:17.743683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:17.743683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:17.743683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:17.743683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:17.744921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:17.744921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:17.744921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:17.744921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:17.744921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:17.744921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:17.744921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:17.744921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:17.744921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:17.744921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:17.744921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:17.744921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:17.744921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:17.744921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:17.744921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:17.744921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:17.744921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:17.745984 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:17.745984 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:17.745984 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:17.745984 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:17.747956 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 233/459] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:17.749476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed13ad4-6f64-4748-8205-0046deb75d1c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:17.749895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed13ad4-6f64-4748-8205-0046deb75d1c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:17.750300 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed13ad4-6f64-4748-8205-0046deb75d1c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:17.751915 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 234/460] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:17.756839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed13ad4-6f64-4748-8205-0046deb75d1c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:17.784157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 227/461] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:54:17.786663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b2b1c-66ff-4d80-94ba-879df2397eda None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:17.787219 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b2b1c-66ff-4d80-94ba-879df2397eda None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:17.787760 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957b2b1c-66ff-4d80-94ba-879df2397eda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:17.810804 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-957b2b1c-66ff-4d80-94ba-879df2397eda None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:17.812145 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-957b2b1c-66ff-4d80-94ba-879df2397eda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:17.897461 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed13ad4-6f64-4748-8205-0046deb75d1c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:17.897778 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 228/462] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:17.898819 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b2b1c-66ff-4d80-94ba-879df2397eda None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:17.945778 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7bdf2-7b1b-4569-b074-01d1a39af8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:17.947170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7bdf2-7b1b-4569-b074-01d1a39af8fe None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:17.947463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7bdf2-7b1b-4569-b074-01d1a39af8fe None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:17.947745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53e7bdf2-7b1b-4569-b074-01d1a39af8fe None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:17.948573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7bdf2-7b1b-4569-b074-01d1a39af8fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:17.948888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7bdf2-7b1b-4569-b074-01d1a39af8fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:17.950888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53e7bdf2-7b1b-4569-b074-01d1a39af8fe None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:17.972010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 235/463] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:17.973220 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7bdf2-7b1b-4569-b074-01d1a39af8fe None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:17.987581 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:18.014852 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:18.023306 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957b2b1c-66ff-4d80-94ba-879df2397eda None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:18.059237 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09575065-e6bd-49a3-b7c0-3fef2a4b7c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:18.060294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09575065-e6bd-49a3-b7c0-3fef2a4b7c98 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:18.060473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09575065-e6bd-49a3-b7c0-3fef2a4b7c98 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:18.060857 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09575065-e6bd-49a3-b7c0-3fef2a4b7c98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:18.076252 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09575065-e6bd-49a3-b7c0-3fef2a4b7c98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:18.077871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09575065-e6bd-49a3-b7c0-3fef2a4b7c98 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:18.097526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-09575065-e6bd-49a3-b7c0-3fef2a4b7c98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:18.097526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:18.097526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:18.097526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:18.097526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:18.097526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:18.097526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:18.097526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:18.097526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:18.097526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:18.097526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:18.097526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:18.097526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:18.097526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:18.097526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:18.098412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:18.098412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:18.098412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:18.098412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:18.098412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:18.098412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:18.098412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:18.098412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:18.098412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:18.098412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:18.098412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:18.098412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:18.098412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:18.098412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:18.098412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:18.098412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:18.098412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:18.099301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:18.099301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:18.099301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:18.099301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:18.099301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 229/464] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:18.099568 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09575065-e6bd-49a3-b7c0-3fef2a4b7c98 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:18.110990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53e7bdf2-7b1b-4569-b074-01d1a39af8fe None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:18.150625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100354b0-fbcb-4f60-a54a-6f094537753a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:18.151351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100354b0-fbcb-4f60-a54a-6f094537753a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:18.151639 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100354b0-fbcb-4f60-a54a-6f094537753a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:18.151953 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100354b0-fbcb-4f60-a54a-6f094537753a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:18.162619 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100354b0-fbcb-4f60-a54a-6f094537753a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:18.164124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100354b0-fbcb-4f60-a54a-6f094537753a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:18.181628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-100354b0-fbcb-4f60-a54a-6f094537753a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:18.181628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:18.181628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:18.181628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:18.181628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:18.181628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:18.181628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:18.181628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:18.181628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:18.181628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:18.181628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:18.181628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:18.181628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:18.181628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:18.181628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:18.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:18.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:18.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:18.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:18.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:18.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:18.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:18.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:18.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:18.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:18.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:18.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:18.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:18.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:18.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:18.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:18.182528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:18.183423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:18.183423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:18.183423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:18.183423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:18.187724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 236/465] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:18.188604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100354b0-fbcb-4f60-a54a-6f094537753a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:18.207900 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09575065-e6bd-49a3-b7c0-3fef2a4b7c98 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:18.244131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e6316ab-13c4-44c1-97b6-b9f14cc506b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:18.244808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6316ab-13c4-44c1-97b6-b9f14cc506b4 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:18.245095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6316ab-13c4-44c1-97b6-b9f14cc506b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:18.245375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e6316ab-13c4-44c1-97b6-b9f14cc506b4 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:18.246205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6316ab-13c4-44c1-97b6-b9f14cc506b4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:18.247674 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e6316ab-13c4-44c1-97b6-b9f14cc506b4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:18.251033 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:18.269497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 230/466] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:54:18 2022] POST /identity/v3/users => generated 246 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:54:18.270942 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c13e3-85c9-4c38-9938-8cc40bb6b25f None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:18.271333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c13e3-85c9-4c38-9938-8cc40bb6b25f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:18.271626 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5c13e3-85c9-4c38-9938-8cc40bb6b25f None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:18.275981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 231/467] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:18.277191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c13e3-85c9-4c38-9938-8cc40bb6b25f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:18.282830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:18.314122 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100354b0-fbcb-4f60-a54a-6f094537753a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:18.362862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-752428ca-e8bc-460a-9612-7480bae0f69b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:18.363653 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752428ca-e8bc-460a-9612-7480bae0f69b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:18.364001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752428ca-e8bc-460a-9612-7480bae0f69b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:18.364285 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-752428ca-e8bc-460a-9612-7480bae0f69b None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:18.365102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752428ca-e8bc-460a-9612-7480bae0f69b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:18.365421 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752428ca-e8bc-460a-9612-7480bae0f69b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:18.366913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-752428ca-e8bc-460a-9612-7480bae0f69b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:18.381194 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 237/468] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:18.382641 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f2018-1f1f-46ca-afa2-527603be36bc None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:18.382915 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f2018-1f1f-46ca-afa2-527603be36bc None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:18.383213 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f94f2018-1f1f-46ca-afa2-527603be36bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:18.401768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f94f2018-1f1f-46ca-afa2-527603be36bc None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:18.402876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f94f2018-1f1f-46ca-afa2-527603be36bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:18.427699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5c13e3-85c9-4c38-9938-8cc40bb6b25f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:18.467287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90c17c-e137-461d-9fdb-be6631b17805 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:18.468091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90c17c-e137-461d-9fdb-be6631b17805 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:18.468428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90c17c-e137-461d-9fdb-be6631b17805 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:18.468712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf90c17c-e137-461d-9fdb-be6631b17805 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:18.469516 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90c17c-e137-461d-9fdb-be6631b17805 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:18.469887 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90c17c-e137-461d-9fdb-be6631b17805 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:18.471710 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf90c17c-e137-461d-9fdb-be6631b17805 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:18.487748 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 232/469] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:18.493099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90c17c-e137-461d-9fdb-be6631b17805 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:18.493827 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 238/470] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:18.497516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f2018-1f1f-46ca-afa2-527603be36bc None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:18.554155 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:18.578728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:18.613605 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f94f2018-1f1f-46ca-afa2-527603be36bc None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:18.623456 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf90c17c-e137-461d-9fdb-be6631b17805 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:18.658017 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3039a-5bad-4736-b9ac-d33cc200a05d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:18.658627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3039a-5bad-4736-b9ac-d33cc200a05d None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:18.658914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3039a-5bad-4736-b9ac-d33cc200a05d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:18.659192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8c3039a-5bad-4736-b9ac-d33cc200a05d None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:18.659951 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3039a-5bad-4736-b9ac-d33cc200a05d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:18.661447 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8c3039a-5bad-4736-b9ac-d33cc200a05d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:18.667187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df7580-b87b-484d-b05a-af4a98d10c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:18.668142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df7580-b87b-484d-b05a-af4a98d10c67 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:18.668620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df7580-b87b-484d-b05a-af4a98d10c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:18.669086 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07df7580-b87b-484d-b05a-af4a98d10c67 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:18.669979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df7580-b87b-484d-b05a-af4a98d10c67 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:18.671689 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07df7580-b87b-484d-b05a-af4a98d10c67 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:18.683521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 239/471] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:54:18 2022] POST /identity/v3/users => generated 248 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:54:18.692927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3039a-5bad-4736-b9ac-d33cc200a05d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:18.718627 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 233/472] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:54:18 2022] POST /identity/v3/users => generated 248 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:54:18.810969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8c3039a-5bad-4736-b9ac-d33cc200a05d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:18.819002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:18.850543 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c9c3f6c-bacf-4d1d-8e49-4846beb371a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:18.851392 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c3f6c-bacf-4d1d-8e49-4846beb371a3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:18.851682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c3f6c-bacf-4d1d-8e49-4846beb371a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:18.852004 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c9c3f6c-bacf-4d1d-8e49-4846beb371a3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:18.865017 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c3f6c-bacf-4d1d-8e49-4846beb371a3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:18.866597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c9c3f6c-bacf-4d1d-8e49-4846beb371a3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:18.885758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4c9c3f6c-bacf-4d1d-8e49-4846beb371a3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:18.885758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:18.885758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:18.885758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:18.885758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:18.885758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:18.885758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:18.885758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:18.885758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:18.885758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:18.885758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:18.885758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:18.885758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:18.885758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:18.885758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:18.886701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:18.886701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:18.886701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:18.886701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:18.886701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:18.886701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:18.886701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:18.886701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:18.886701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:18.886701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:18.886701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:18.886701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:18.886701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:18.886701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:18.886701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:18.886701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:18.886701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:18.887788 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:18.887788 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:18.887788 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:18.887788 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:18.887788 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 240/473] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:18.894415 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df7580-b87b-484d-b05a-af4a98d10c67 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:18.973614 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed204de2-258a-487c-a227-f34f0bcddbf5 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:18.973826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed204de2-258a-487c-a227-f34f0bcddbf5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:18.974109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed204de2-258a-487c-a227-f34f0bcddbf5 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:18.975674 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 241/474] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:18.992456 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacdf9d-d8bf-4ede-979a-be46d174d1ce None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:18.992745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacdf9d-d8bf-4ede-979a-be46d174d1ce None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:18.993026 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aacdf9d-d8bf-4ede-979a-be46d174d1ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:19.019332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0aacdf9d-d8bf-4ede-979a-be46d174d1ce None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:19.021140 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0aacdf9d-d8bf-4ede-979a-be46d174d1ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:19.039135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07df7580-b87b-484d-b05a-af4a98d10c67 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:19.063199 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:19.086715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca290e-fea5-489b-bc2e-120a839252fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:19.087405 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca290e-fea5-489b-bc2e-120a839252fa None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:19.088240 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca290e-fea5-489b-bc2e-120a839252fa None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:19.088531 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9ca290e-fea5-489b-bc2e-120a839252fa None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:19.089277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca290e-fea5-489b-bc2e-120a839252fa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:19.089591 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca290e-fea5-489b-bc2e-120a839252fa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:19.090983 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9ca290e-fea5-489b-bc2e-120a839252fa None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:19.105516 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 234/475] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:19.111110 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca290e-fea5-489b-bc2e-120a839252fa None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:19.118512 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 242/476] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:54:19.142266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacdf9d-d8bf-4ede-979a-be46d174d1ce None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:19.234263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9ca290e-fea5-489b-bc2e-120a839252fa None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:19.258175 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:19.277541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aacdf9d-d8bf-4ede-979a-be46d174d1ce None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:19.279602 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47e7c1-cb50-42ae-8050-4c29c5c24552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:19.280323 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47e7c1-cb50-42ae-8050-4c29c5c24552 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:19.280613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47e7c1-cb50-42ae-8050-4c29c5c24552 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:19.280892 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce47e7c1-cb50-42ae-8050-4c29c5c24552 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:19.281613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47e7c1-cb50-42ae-8050-4c29c5c24552 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:19.283078 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce47e7c1-cb50-42ae-8050-4c29c5c24552 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:19.308880 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 235/477] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:54:19 2022] POST /identity/v3/users => generated 249 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:54:19.327426 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b921b-fb4c-40ee-a5a3-5a97392118ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:19.328579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b921b-fb4c-40ee-a5a3-5a97392118ec None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:19.329042 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b921b-fb4c-40ee-a5a3-5a97392118ec None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:19.329497 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005b921b-fb4c-40ee-a5a3-5a97392118ec None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:19.330464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b921b-fb4c-40ee-a5a3-5a97392118ec None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:19.332151 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005b921b-fb4c-40ee-a5a3-5a97392118ec None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:19.342216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.sql.core [None req-005b921b-fb4c-40ee-a5a3-5a97392118ec None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 15:54:19.342216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 25 15:54:19.342216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [parameters: {'id': '71fa5bab618843d6a099952255eb5509', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 25 15:54:19.342216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80233) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 15:54:19.360831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-005b921b-fb4c-40ee-a5a3-5a97392118ec None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 25 15:54:19.360831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:19.360831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 15:54:19.360831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 15:54:19.360831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 15:54:19.360831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 15:54:19.360831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 15:54:19.360831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result = self._query(query) May 25 15:54:19.360831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 15:54:19.360831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application conn.query(q) May 25 15:54:19.360831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 15:54:19.360831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 15:54:19.360831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 15:54:19.360831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result.read() May 25 15:54:19.360831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return ref.to_dict() May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 15:54:19.361799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application next(self.gen) May 25 15:54:19.362729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 15:54:19.362729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application yield resource May 25 15:54:19.362729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 15:54:19.362729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application next(self.gen) May 25 15:54:19.362729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 15:54:19.362729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.session.rollback() May 25 15:54:19.362729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 15:54:19.362729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.force_reraise() May 25 15:54:19.362729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 15:54:19.362729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise self.value May 25 15:54:19.362729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 15:54:19.362729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 15:54:19.362729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 15:54:19.362729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application session.commit() May 25 15:54:19.362729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 15:54:19.362729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 15:54:19.362729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 15:54:19.363625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._prepare_impl() May 25 15:54:19.363625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 15:54:19.363625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.session.flush() May 25 15:54:19.363625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 15:54:19.363625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._flush(objects) May 25 15:54:19.363625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 15:54:19.363625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 15:54:19.363625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 15:54:19.363625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application compat.raise_( May 25 15:54:19.363625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 15:54:19.363625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception May 25 15:54:19.363625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 15:54:19.363625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application flush_context.execute() May 25 15:54:19.363625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 15:54:19.363625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rec.execute(self) May 25 15:54:19.363625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 15:54:19.363625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 15:54:19.365000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 15:54:19.365000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 15:54:19.365000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 25 15:54:19.365000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application c = connection._execute_20( May 25 15:54:19.365000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 15:54:19.365000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 15:54:19.365000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 15:54:19.365000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 15:54:19.365000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 15:54:19.365000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 15:54:19.365000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 15:54:19.365000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 15:54:19.365000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 15:54:19.365000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 15:54:19.365000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 15:54:19.365000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception May 25 15:54:19.366034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 15:54:19.366034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 15:54:19.366034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 15:54:19.366034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 15:54:19.366034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 15:54:19.366034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result = self._query(query) May 25 15:54:19.366034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 15:54:19.366034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application conn.query(q) May 25 15:54:19.366034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 15:54:19.366034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 15:54:19.366034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 15:54:19.366034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application result.read() May 25 15:54:19.366034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 15:54:19.366034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 15:54:19.366034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 15:54:19.366034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 15:54:19.367038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 15:54:19.367038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 15:54:19.367038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 15:54:19.367038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 15:54:19.367038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") May 25 15:54:19.367038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] May 25 15:54:19.367038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [parameters: {'id': '71fa5bab618843d6a099952255eb5509', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] May 25 15:54:19.367038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 15:54:19.367038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:19.367038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:54:19.367038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:19.367038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:19.367038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:19.367038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:19.367038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:19.367038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:19.368040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:19.368040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:19.368040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:19.368040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:19.368040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:19.368040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:19.368040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post May 25 15:54:19.368040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( May 25 15:54:19.368040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:19.368040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:19.368040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role May 25 15:54:19.368040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) May 25 15:54:19.368040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 15:54:19.368040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 15:54:19.368040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. May 25 15:54:19.368040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:19.371064 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 243/478] 10.222.0.85 () {60 vars in 1131 bytes} [Wed May 25 15:54:19 2022] POST /identity/v3/roles => generated 144 bytes in 229 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 25 15:54:19.378858 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47e7c1-cb50-42ae-8050-4c29c5c24552 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:19.440558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:19.536171 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce47e7c1-cb50-42ae-8050-4c29c5c24552 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:19.581140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75929b60-d3db-45e9-9636-cb547788231f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:19.581980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75929b60-d3db-45e9-9636-cb547788231f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:19.582266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75929b60-d3db-45e9-9636-cb547788231f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:19.582544 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75929b60-d3db-45e9-9636-cb547788231f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:19.595690 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75929b60-d3db-45e9-9636-cb547788231f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:19.604797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-75929b60-d3db-45e9-9636-cb547788231f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:54:19.604797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:19.604797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:19.604797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:19.604797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:19.604797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:19.604797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:19.604797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:19.604797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:19.604797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:19.604797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:19.604797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:19.604797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:19.604797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:19.604797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:19.604797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:19.605736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:19.605736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:19.605736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:19.605736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:19.605736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:19.605736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:19.605736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:19.605736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:19.605736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:19.605736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:19.605736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:19.605736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:19.605736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:19.605736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:19.605736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:19.605736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:19.606560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:19.606560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:19.606560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:19.606560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:19.606560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:19.606560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:19.606560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:19.606560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:19.606560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:19.606560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:19.606560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. May 25 15:54:19.606560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:19.606560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 236/479] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:54:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:19.612213 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b921b-fb4c-40ee-a5a3-5a97392118ec None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:19.716526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005b921b-fb4c-40ee-a5a3-5a97392118ec None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:19.752064 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12aadc2e-74bb-4dc8-8098-2c309a042938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:19.752780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aadc2e-74bb-4dc8-8098-2c309a042938 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:19.753075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aadc2e-74bb-4dc8-8098-2c309a042938 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:19.753363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12aadc2e-74bb-4dc8-8098-2c309a042938 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:19.754158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aadc2e-74bb-4dc8-8098-2c309a042938 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:19.754488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aadc2e-74bb-4dc8-8098-2c309a042938 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:19.755950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12aadc2e-74bb-4dc8-8098-2c309a042938 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:19.765427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 244/480] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:21.006535 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28cc55-dd0a-4b48-94d1-c957edf2b282 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:21.007063 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28cc55-dd0a-4b48-94d1-c957edf2b282 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:21.007063 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28cc55-dd0a-4b48-94d1-c957edf2b282 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:21.008790 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 237/481] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:21.014301 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed80ea8-e910-499e-8293-eee92ec31074 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:21.014716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed80ea8-e910-499e-8293-eee92ec31074 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:21.014885 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed80ea8-e910-499e-8293-eee92ec31074 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:21.032725 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7ed80ea8-e910-499e-8293-eee92ec31074 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:21.033982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7ed80ea8-e910-499e-8293-eee92ec31074 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:21.044629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:21.126936 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 245/482] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:21.145046 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28cc55-dd0a-4b48-94d1-c957edf2b282 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:21.279484 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:21.294192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28cc55-dd0a-4b48-94d1-c957edf2b282 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:21.342957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c8ed36-5d64-430f-a341-1f91c19c06b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:21.343780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8ed36-5d64-430f-a341-1f91c19c06b6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:21.344091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8ed36-5d64-430f-a341-1f91c19c06b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:21.344370 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c8ed36-5d64-430f-a341-1f91c19c06b6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:21.366653 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c8ed36-5d64-430f-a341-1f91c19c06b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:21.374849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-c9c8ed36-5d64-430f-a341-1f91c19c06b6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:21.374849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:21.374849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:21.374849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:21.374849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:21.374849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:21.374849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:21.374849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:21.374849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:21.374849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:21.374849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:21.374849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:21.374849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:21.374849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:21.374849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:21.374849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:21.375845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:21.375845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:21.375845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:21.375845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:21.375845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:21.375845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:21.375845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:21.375845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:21.375845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:21.375845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:21.375845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:21.375845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:21.375845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:21.375845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:21.375845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:21.375845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:21.376661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:21.376661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:21.376661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:21.376661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:21.376661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:21.376661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:21.376661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:21.376661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:21.376661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:21.376661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:21.376661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:21.376661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:21.376661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 238/483] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:21.382409 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed80ea8-e910-499e-8293-eee92ec31074 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:21.452481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c273053-2e9e-4734-992d-5f3e1785fb23 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:21.452787 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c273053-2e9e-4734-992d-5f3e1785fb23 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:21.453053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c273053-2e9e-4734-992d-5f3e1785fb23 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:21.454804 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 239/484] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:21.460498 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66978e-301b-451c-aa33-d2eea6f06622 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:21.460792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66978e-301b-451c-aa33-d2eea6f06622 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:21.461078 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d66978e-301b-451c-aa33-d2eea6f06622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:21.480989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6d66978e-301b-451c-aa33-d2eea6f06622 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:21.482180 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6d66978e-301b-451c-aa33-d2eea6f06622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:21.569200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed80ea8-e910-499e-8293-eee92ec31074 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:21.626120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 240/485] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:54:21.627712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e9a3-def6-4e36-a840-15c7cc420182 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:21.628033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e9a3-def6-4e36-a840-15c7cc420182 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:21.628320 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0398e9a3-def6-4e36-a840-15c7cc420182 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:21.629725 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 241/486] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:21.634814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301e08b-f0ab-4c4b-85b6-f822718e9de5 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:21.635109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301e08b-f0ab-4c4b-85b6-f822718e9de5 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:21.635394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b301e08b-f0ab-4c4b-85b6-f822718e9de5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:21.635745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a310b6-e501-4255-8fb8-7fde91c6e1a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:21.636762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a310b6-e501-4255-8fb8-7fde91c6e1a5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:21.637350 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a310b6-e501-4255-8fb8-7fde91c6e1a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:21.637901 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a310b6-e501-4255-8fb8-7fde91c6e1a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:21.639114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a310b6-e501-4255-8fb8-7fde91c6e1a5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:21.639735 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a310b6-e501-4255-8fb8-7fde91c6e1a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:21.641371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a310b6-e501-4255-8fb8-7fde91c6e1a5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:21.654989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 246/487] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:21.655950 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b301e08b-f0ab-4c4b-85b6-f822718e9de5 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:21.656879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a310b6-e501-4255-8fb8-7fde91c6e1a5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:21.657116 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b301e08b-f0ab-4c4b-85b6-f822718e9de5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:21.666824 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:21.741305 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 242/488] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:54:21.742183 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301e08b-f0ab-4c4b-85b6-f822718e9de5 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:21.762864 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a310b6-e501-4255-8fb8-7fde91c6e1a5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:21.771127 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:21.798166 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7067a-a6c0-4d27-8aea-ac1039d6af35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:21.799350 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7067a-a6c0-4d27-8aea-ac1039d6af35 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:21.799874 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7067a-a6c0-4d27-8aea-ac1039d6af35 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:21.800358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a7067a-a6c0-4d27-8aea-ac1039d6af35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:21.811779 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7067a-a6c0-4d27-8aea-ac1039d6af35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:21.813506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a7067a-a6c0-4d27-8aea-ac1039d6af35 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:21.830990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-34a7067a-a6c0-4d27-8aea-ac1039d6af35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:21.830990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:21.830990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:21.830990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:21.830990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:21.830990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:21.830990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:21.830990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:21.830990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:21.830990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:21.830990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:21.830990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:21.830990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:21.830990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:21.830990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:21.831866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:21.831866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:21.831866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:21.831866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:21.831866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:21.831866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:21.831866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:21.831866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:21.831866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:21.831866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:21.831866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:21.831866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:21.831866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:21.831866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:21.831866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:21.831866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:21.831866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:21.832745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:21.832745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:21.832745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:21.832745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:21.839500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 247/489] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:21.841142 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7067a-a6c0-4d27-8aea-ac1039d6af35 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:21.887312 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b301e08b-f0ab-4c4b-85b6-f822718e9de5 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:21.902261 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:21.929080 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3e642-f69c-48fd-89b8-97ffd187ffe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:21.929997 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3e642-f69c-48fd-89b8-97ffd187ffe8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:21.930296 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3e642-f69c-48fd-89b8-97ffd187ffe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:21.930579 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f3e642-f69c-48fd-89b8-97ffd187ffe8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:21.944117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f3e642-f69c-48fd-89b8-97ffd187ffe8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:21.946920 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a7067a-a6c0-4d27-8aea-ac1039d6af35 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:21.952458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-42f3e642-f69c-48fd-89b8-97ffd187ffe8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:21.952458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:21.952458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:21.952458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:21.952458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:21.952458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:21.952458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:21.952458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:21.952458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:21.952458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:21.952458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:21.952458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:21.952458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:21.952458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:21.952458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:21.952458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:21.953379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:21.953379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:21.953379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:21.953379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:21.953379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:21.953379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:21.953379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:21.953379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:21.953379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:21.953379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:21.953379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:21.953379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:21.953379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:21.953379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:21.953379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:21.953379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:21.954609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:21.954609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:21.954609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:21.954609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:21.954609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:21.954609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:21.954609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:21.954609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:21.954609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:21.954609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:21.954609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:21.954609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:21.954609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 243/490] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:21.955569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3e642-f69c-48fd-89b8-97ffd187ffe8 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:21.985550 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35354008-b9a5-4ba0-ab9d-0360ed642aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:21.986732 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35354008-b9a5-4ba0-ab9d-0360ed642aee None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:21.987208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35354008-b9a5-4ba0-ab9d-0360ed642aee None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:21.987671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35354008-b9a5-4ba0-ab9d-0360ed642aee None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:21.998444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35354008-b9a5-4ba0-ab9d-0360ed642aee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:22.006566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-35354008-b9a5-4ba0-ab9d-0360ed642aee None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:22.006566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:22.006566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:22.006566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:22.006566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:22.006566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:22.006566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:22.006566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:22.006566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:22.006566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:22.006566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:22.006566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:22.006566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:22.006566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:22.006566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:22.006566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:22.007444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:22.007444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:22.007444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:22.007444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:22.007444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:22.007444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:22.007444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:22.007444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:22.007444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:22.007444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:22.007444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:22.007444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:22.007444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:22.007444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:22.007444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:22.007444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:22.008604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:22.008604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:22.008604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:22.008604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:22.008604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:22.008604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:22.008604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:22.008604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:22.008604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:22.008604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:22.008604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:22.008604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:22.011046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 248/491] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:21 2022] GET /identity/v3/roles/service => generated 85 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:22.012060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35354008-b9a5-4ba0-ab9d-0360ed642aee None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:22.044721 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:22.073699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f3e642-f69c-48fd-89b8-97ffd187ffe8 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:22.110932 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56133873-87ab-4357-bfcc-adf4942a02f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:22.111527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56133873-87ab-4357-bfcc-adf4942a02f6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:22.111833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56133873-87ab-4357-bfcc-adf4942a02f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:22.112126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56133873-87ab-4357-bfcc-adf4942a02f6 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:22.112986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56133873-87ab-4357-bfcc-adf4942a02f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:22.113307 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56133873-87ab-4357-bfcc-adf4942a02f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:22.114812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56133873-87ab-4357-bfcc-adf4942a02f6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:22.117021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35354008-b9a5-4ba0-ab9d-0360ed642aee None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:22.132668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 244/492] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:22.133694 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56133873-87ab-4357-bfcc-adf4942a02f6 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:22.136066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:22.155886 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a30628-6e66-4376-8707-a81ac1af114c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:22.156734 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a30628-6e66-4376-8707-a81ac1af114c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:22.157110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a30628-6e66-4376-8707-a81ac1af114c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:22.157496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a30628-6e66-4376-8707-a81ac1af114c None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:22.158376 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a30628-6e66-4376-8707-a81ac1af114c None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:22.158816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a30628-6e66-4376-8707-a81ac1af114c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:22.160564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a30628-6e66-4376-8707-a81ac1af114c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:22.170331 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 249/493] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:22.171497 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a30628-6e66-4376-8707-a81ac1af114c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:22.255843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56133873-87ab-4357-bfcc-adf4942a02f6 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:22.273183 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a30628-6e66-4376-8707-a81ac1af114c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:22.293793 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a286238-ddc3-4d4f-baaf-3851e1e9d5fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:22.294873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a286238-ddc3-4d4f-baaf-3851e1e9d5fb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:22.295308 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a286238-ddc3-4d4f-baaf-3851e1e9d5fb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:22.295733 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a286238-ddc3-4d4f-baaf-3851e1e9d5fb None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:22.296752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a286238-ddc3-4d4f-baaf-3851e1e9d5fb None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:22.297262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a286238-ddc3-4d4f-baaf-3851e1e9d5fb None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:22.298980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a286238-ddc3-4d4f-baaf-3851e1e9d5fb None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:22.300847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:22.308936 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f06b2f-a542-4bf0-b61e-a32bbe17929a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:22.309819 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f06b2f-a542-4bf0-b61e-a32bbe17929a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:22.314627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f06b2f-a542-4bf0-b61e-a32bbe17929a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:22.314627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f06b2f-a542-4bf0-b61e-a32bbe17929a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:22.316576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 245/494] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:22.317728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a286238-ddc3-4d4f-baaf-3851e1e9d5fb None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:22.319184 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f06b2f-a542-4bf0-b61e-a32bbe17929a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:22.320758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f06b2f-a542-4bf0-b61e-a32bbe17929a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:22.336775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e5f06b2f-a542-4bf0-b61e-a32bbe17929a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 15:54:22.336775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:22.336775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:22.336775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:22.336775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:22.336775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:22.336775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:22.336775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:22.336775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:22.336775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:22.336775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:22.336775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:22.336775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:22.336775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:22.336775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:22.336775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:22.337761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:22.338680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:22.338680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:22.338680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:22.338680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:22.338680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:22.338680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:22.338680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:22.338680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:22.338680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 15:54:22.338680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:22.342720 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 250/495] 10.222.0.85 () {56 vars in 1093 bytes} [Wed May 25 15:54:22 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:22.343669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f06b2f-a542-4bf0-b61e-a32bbe17929a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:22.387791 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:22.428357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a286238-ddc3-4d4f-baaf-3851e1e9d5fb None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:22.453376 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f06b2f-a542-4bf0-b61e-a32bbe17929a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:22.465066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abdca9a-b23b-4ed1-adf2-42c1dccb9d78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:22.465881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdca9a-b23b-4ed1-adf2-42c1dccb9d78 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:22.466287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdca9a-b23b-4ed1-adf2-42c1dccb9d78 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:22.466469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abdca9a-b23b-4ed1-adf2-42c1dccb9d78 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:22.478800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abdca9a-b23b-4ed1-adf2-42c1dccb9d78 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:22.480374 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abdca9a-b23b-4ed1-adf2-42c1dccb9d78 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:22.492349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f4d5d-332c-495e-88ba-f982694a087c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:22.493159 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f4d5d-332c-495e-88ba-f982694a087c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:22.493446 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f4d5d-332c-495e-88ba-f982694a087c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:22.493730 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f2f4d5d-332c-495e-88ba-f982694a087c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:22.502037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-1abdca9a-b23b-4ed1-adf2-42c1dccb9d78 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:22.502037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:22.502037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:22.502037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:22.502037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:22.502037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:22.502037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:22.502037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:22.502037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:22.502037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:22.502037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:22.502037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:22.502037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:22.502037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:22.502037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:22.503188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:22.503188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:22.503188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:22.503188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:22.503188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:22.503188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:22.503188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:22.503188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:22.503188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:22.503188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:22.503188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:22.503188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:22.503188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:22.503188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:22.503188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:22.503188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:22.503188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:22.504517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:22.504517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:22.504517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:22.504517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:22.504517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 246/496] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:22.504962 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a764341-dc1c-4b1b-a846-6829ca700a7a None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:22.505668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a764341-dc1c-4b1b-a846-6829ca700a7a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:22.505668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a764341-dc1c-4b1b-a846-6829ca700a7a None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:22.506989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 247/497] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:22.508771 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a764341-dc1c-4b1b-a846-6829ca700a7a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:22.511262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f4d5d-332c-495e-88ba-f982694a087c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:22.512811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f2f4d5d-332c-495e-88ba-f982694a087c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:22.531560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4f2f4d5d-332c-495e-88ba-f982694a087c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:22.531560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:22.531560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:22.531560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:22.531560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:22.531560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:22.531560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:22.531560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:22.531560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:22.531560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:22.531560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:22.531560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:22.531560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:22.531560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:22.531560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:22.532700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:22.532700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:22.532700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:22.532700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:22.532700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:22.532700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:22.532700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:22.532700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:22.532700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:22.532700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:22.532700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:22.532700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:22.532700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:22.532700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:22.532700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:22.532700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:22.532700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:22.533561 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:22.533561 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:22.533561 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:22.533561 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:22.533561 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 251/498] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:22.534455 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f4d5d-332c-495e-88ba-f982694a087c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:22.569286 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:22.627188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a764341-dc1c-4b1b-a846-6829ca700a7a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:22.639838 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:22.649585 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f2f4d5d-332c-495e-88ba-f982694a087c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:22.662117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e4d43-45e0-4d74-aa0b-2df9a7110222 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:22.662797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e4d43-45e0-4d74-aa0b-2df9a7110222 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:22.663089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e4d43-45e0-4d74-aa0b-2df9a7110222 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:22.663369 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90e4d43-45e0-4d74-aa0b-2df9a7110222 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:22.664262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e4d43-45e0-4d74-aa0b-2df9a7110222 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:22.664579 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e4d43-45e0-4d74-aa0b-2df9a7110222 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:22.666029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90e4d43-45e0-4d74-aa0b-2df9a7110222 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:22.682577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01ef1abf-7c7a-4625-96d8-29bf417abb76 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:22.683237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef1abf-7c7a-4625-96d8-29bf417abb76 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:22.683521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef1abf-7c7a-4625-96d8-29bf417abb76 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:22.683796 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01ef1abf-7c7a-4625-96d8-29bf417abb76 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:22.684544 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef1abf-7c7a-4625-96d8-29bf417abb76 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:22.684852 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef1abf-7c7a-4625-96d8-29bf417abb76 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:22.686284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01ef1abf-7c7a-4625-96d8-29bf417abb76 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:22.697656 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 252/499] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:22.698906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce39e4-f3c4-4b2b-b035-984911b55bb1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:22.699197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce39e4-f3c4-4b2b-b035-984911b55bb1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:22.699481 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ce39e4-f3c4-4b2b-b035-984911b55bb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:22.702165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 248/500] 10.222.0.85 () {56 vars in 1133 bytes} [Wed May 25 15:54:22 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:22.703005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e4d43-45e0-4d74-aa0b-2df9a7110222 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:22.717223 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e6ce39e4-f3c4-4b2b-b035-984911b55bb1 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:22.718442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e6ce39e4-f3c4-4b2b-b035-984911b55bb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:22.782091 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 253/501] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:22.789876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fe914-8bdf-4caf-8042-817a4c7531ee None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:22.789876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fe914-8bdf-4caf-8042-817a4c7531ee None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:22.789876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e8fe914-8bdf-4caf-8042-817a4c7531ee None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:22.789876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 254/502] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:22.789876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fe914-8bdf-4caf-8042-817a4c7531ee None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:22.809477 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90e4d43-45e0-4d74-aa0b-2df9a7110222 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:22.816840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:22.846483 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada12a7-207c-44ca-a041-7aa5c6532ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:22.847150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada12a7-207c-44ca-a041-7aa5c6532ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:22.847436 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada12a7-207c-44ca-a041-7aa5c6532ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:22.847709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ada12a7-207c-44ca-a041-7aa5c6532ca6 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:22.848471 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada12a7-207c-44ca-a041-7aa5c6532ca6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:22.848778 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada12a7-207c-44ca-a041-7aa5c6532ca6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:22.850173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ada12a7-207c-44ca-a041-7aa5c6532ca6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:22.861574 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 249/503] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:22.862371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada12a7-207c-44ca-a041-7aa5c6532ca6 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:22.872083 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:22.891286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e8fe914-8bdf-4caf-8042-817a4c7531ee None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:22.924861 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45f8d882-8791-475d-916f-530c37d8b16e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:22.925674 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8d882-8791-475d-916f-530c37d8b16e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:22.925941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8d882-8791-475d-916f-530c37d8b16e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:22.926224 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f8d882-8791-475d-916f-530c37d8b16e None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:22.933558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f8d882-8791-475d-916f-530c37d8b16e None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:22.935032 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45f8d882-8791-475d-916f-530c37d8b16e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:22.949212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-45f8d882-8791-475d-916f-530c37d8b16e None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 15:54:22.949212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:22.949212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:22.949212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:22.949212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:22.949212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:22.949212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:22.949212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:22.949212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:22.949212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:22.949212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:22.949212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:22.949212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:22.949212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:22.949212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:22.949212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:22.950305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:22.951292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:22.951292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:22.951292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:22.951292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:22.951292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:22.951292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:22.951292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:22.951292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:22.951292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 15:54:22.951292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:22.951292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 255/504] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:22 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:22.951923 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab1fb3-973f-4d32-9827-841eb058ac49 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:22.952155 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab1fb3-973f-4d32-9827-841eb058ac49 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:22.952371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ab1fb3-973f-4d32-9827-841eb058ac49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:22.968443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c2ab1fb3-973f-4d32-9827-841eb058ac49 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:22.969223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ada12a7-207c-44ca-a041-7aa5c6532ca6 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:22.969620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c2ab1fb3-973f-4d32-9827-841eb058ac49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:23.003037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61606c77-caae-4d47-9d92-6901a6da71cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:23.003695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61606c77-caae-4d47-9d92-6901a6da71cd None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:23.004022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61606c77-caae-4d47-9d92-6901a6da71cd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:23.004308 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61606c77-caae-4d47-9d92-6901a6da71cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:23.035391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 256/505] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:23.036251 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab1fb3-973f-4d32-9827-841eb058ac49 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:23.037175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:23.073636 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61606c77-caae-4d47-9d92-6901a6da71cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:23.075126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61606c77-caae-4d47-9d92-6901a6da71cd None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:23.115285 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:23.146273 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ab1fb3-973f-4d32-9827-841eb058ac49 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:23.173354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 250/506] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:54:23.174193 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61606c77-caae-4d47-9d92-6901a6da71cd None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:23.180980 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492cd6ba-de0d-4146-8f0f-04ec8bfae032 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:23.181795 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cd6ba-de0d-4146-8f0f-04ec8bfae032 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:23.182090 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cd6ba-de0d-4146-8f0f-04ec8bfae032 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:23.182375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492cd6ba-de0d-4146-8f0f-04ec8bfae032 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:23.192776 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492cd6ba-de0d-4146-8f0f-04ec8bfae032 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:23.200571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-492cd6ba-de0d-4146-8f0f-04ec8bfae032 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:23.200571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:23.200571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:23.200571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:23.200571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:23.200571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:23.200571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:23.200571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:23.200571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:23.200571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:23.200571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:23.200571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:23.200571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:23.200571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:23.200571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:23.200571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:23.201560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:23.201560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:23.201560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:23.201560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:23.201560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:23.201560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:23.201560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:23.201560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:23.201560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:23.201560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:23.201560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:23.201560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:23.201560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:23.201560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:23.201560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:23.201560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:23.202377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:23.202377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:23.202377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:23.202377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:23.202377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:23.202377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:23.202377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:23.202377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:23.202377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:23.202377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:23.202377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:23.202377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:23.202377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 257/507] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:23 2022] GET /identity/v3/roles/service => generated 85 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:23.203070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492cd6ba-de0d-4146-8f0f-04ec8bfae032 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:23.245160 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:23.279834 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61606c77-caae-4d47-9d92-6901a6da71cd None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:23.306345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492cd6ba-de0d-4146-8f0f-04ec8bfae032 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:23.316421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3215a0-fd3c-4572-8d50-66b1c5a96cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:23.317246 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3215a0-fd3c-4572-8d50-66b1c5a96cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:23.317541 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3215a0-fd3c-4572-8d50-66b1c5a96cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:23.317825 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3215a0-fd3c-4572-8d50-66b1c5a96cd5 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:23.325214 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3215a0-fd3c-4572-8d50-66b1c5a96cd5 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:23.326733 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3215a0-fd3c-4572-8d50-66b1c5a96cd5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:23.337933 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:23.343432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-fa3215a0-fd3c-4572-8d50-66b1c5a96cd5 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 15:54:23.343432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:23.343432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:23.343432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:23.343432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:23.343432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:23.343432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:23.343432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:23.343432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:23.343432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:23.343432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:23.343432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:23.343432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:23.343432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:23.343432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:23.343432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:23.344597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:23.346043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:23.346043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:23.346043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:23.346043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:23.346043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:23.346043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:23.346043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:23.346043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:23.346043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 15:54:23.346043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:23.346572 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4afe9c-827a-43a2-bc33-f1379293e630 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:23.346572 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4afe9c-827a-43a2-bc33-f1379293e630 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:23.346572 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4afe9c-827a-43a2-bc33-f1379293e630 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:23.346762 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 251/508] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:23 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:23.346836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c4afe9c-827a-43a2-bc33-f1379293e630 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:23.346903 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3215a0-fd3c-4572-8d50-66b1c5a96cd5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:23.346969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4afe9c-827a-43a2-bc33-f1379293e630 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:23.346969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4afe9c-827a-43a2-bc33-f1379293e630 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:23.347960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c4afe9c-827a-43a2-bc33-f1379293e630 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:23.384756 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 258/509] 10.222.0.85 () {56 vars in 1137 bytes} [Wed May 25 15:54:23 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:23.385592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4afe9c-827a-43a2-bc33-f1379293e630 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:23.454104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3215a0-fd3c-4572-8d50-66b1c5a96cd5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:23.478959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:23.484283 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c4afe9c-827a-43a2-bc33-f1379293e630 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:23.487654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac01cef-78c6-4f1c-b047-7109eb952b0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:23.488484 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac01cef-78c6-4f1c-b047-7109eb952b0d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:23.488795 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac01cef-78c6-4f1c-b047-7109eb952b0d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:23.489045 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac01cef-78c6-4f1c-b047-7109eb952b0d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:23.499541 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac01cef-78c6-4f1c-b047-7109eb952b0d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:23.501062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac01cef-78c6-4f1c-b047-7109eb952b0d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:23.517933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-3ac01cef-78c6-4f1c-b047-7109eb952b0d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:23.517933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:23.517933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:23.517933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:23.517933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:23.517933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:23.517933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:23.517933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:23.517933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:23.517933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:23.517933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:23.517933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:23.517933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:23.517933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:23.517933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:23.518783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:23.518783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:23.518783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:23.518783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:23.518783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:23.518783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:23.518783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:23.518783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:23.518783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:23.518783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:23.518783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:23.518783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:23.518783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:23.518783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:23.518783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:23.518783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:23.518783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:23.519651 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:23.519651 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:23.519651 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:23.519651 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:23.519903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f718d-f24e-4653-ada0-b7bd673c3f32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:23.520005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 252/510] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:23.520074 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f718d-f24e-4653-ada0-b7bd673c3f32 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:23.520139 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac01cef-78c6-4f1c-b047-7109eb952b0d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:23.520203 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f718d-f24e-4653-ada0-b7bd673c3f32 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:23.520203 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11f718d-f24e-4653-ada0-b7bd673c3f32 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:23.534469 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f718d-f24e-4653-ada0-b7bd673c3f32 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:23.536015 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11f718d-f24e-4653-ada0-b7bd673c3f32 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:23.554657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b11f718d-f24e-4653-ada0-b7bd673c3f32 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:23.554657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:23.554657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:23.554657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:23.554657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:23.554657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:23.554657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:23.554657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:23.554657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:23.554657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:23.554657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:23.554657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:23.554657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:23.554657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:23.554657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:23.555495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:23.555495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:23.555495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:23.555495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:23.555495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:23.555495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:23.555495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:23.555495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:23.555495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:23.555495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:23.555495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:23.555495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:23.555495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:23.555495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:23.555495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:23.555495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:23.555495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:23.556786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:23.556786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:23.556786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:23.556786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:23.556786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 259/511] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:23.556786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f718d-f24e-4653-ada0-b7bd673c3f32 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:23.601562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:23.629760 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac01cef-78c6-4f1c-b047-7109eb952b0d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:23.657046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11f718d-f24e-4653-ada0-b7bd673c3f32 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:23.662061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f95ec5-a678-4739-ba73-fcce7b4d44a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:23.662737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f95ec5-a678-4739-ba73-fcce7b4d44a0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:23.663026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f95ec5-a678-4739-ba73-fcce7b4d44a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:23.663308 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f95ec5-a678-4739-ba73-fcce7b4d44a0 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:23.664094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f95ec5-a678-4739-ba73-fcce7b4d44a0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:23.664409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f95ec5-a678-4739-ba73-fcce7b4d44a0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:23.665825 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03f95ec5-a678-4739-ba73-fcce7b4d44a0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:23.674040 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 253/512] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:23 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:23.674829 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f95ec5-a678-4739-ba73-fcce7b4d44a0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:23.690370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111db553-ad69-4fa7-9ce4-e0d89fd34792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:23.691044 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111db553-ad69-4fa7-9ce4-e0d89fd34792 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:23.691315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111db553-ad69-4fa7-9ce4-e0d89fd34792 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:23.691616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-111db553-ad69-4fa7-9ce4-e0d89fd34792 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:23.692485 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111db553-ad69-4fa7-9ce4-e0d89fd34792 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:23.692801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111db553-ad69-4fa7-9ce4-e0d89fd34792 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:23.694233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-111db553-ad69-4fa7-9ce4-e0d89fd34792 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:23.711676 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 260/513] 10.222.0.85 () {56 vars in 1137 bytes} [Wed May 25 15:54:23 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:23.712660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111db553-ad69-4fa7-9ce4-e0d89fd34792 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:23.718190 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:23.775364 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f95ec5-a678-4739-ba73-fcce7b4d44a0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:23.807377 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7976a-3dd8-49a5-9b03-301e8bc70e48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:23.808053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7976a-3dd8-49a5-9b03-301e8bc70e48 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:23.808338 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7976a-3dd8-49a5-9b03-301e8bc70e48 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:23.808614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c7976a-3dd8-49a5-9b03-301e8bc70e48 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:23.809867 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:23.813259 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-111db553-ad69-4fa7-9ce4-e0d89fd34792 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:23.850131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e12e7-84a4-47db-b73f-5f90ee1cb367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:23.850782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e12e7-84a4-47db-b73f-5f90ee1cb367 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:23.851068 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e12e7-84a4-47db-b73f-5f90ee1cb367 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:23.851348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-580e12e7-84a4-47db-b73f-5f90ee1cb367 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:23.852111 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e12e7-84a4-47db-b73f-5f90ee1cb367 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:23.852426 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e12e7-84a4-47db-b73f-5f90ee1cb367 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:23.853811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-580e12e7-84a4-47db-b73f-5f90ee1cb367 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:23.865074 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 261/514] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:23.865859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e12e7-84a4-47db-b73f-5f90ee1cb367 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:23.876349 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7976a-3dd8-49a5-9b03-301e8bc70e48 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:23.877832 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c7976a-3dd8-49a5-9b03-301e8bc70e48 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:23.916792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:23.965027 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-580e12e7-84a4-47db-b73f-5f90ee1cb367 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:23.972968 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 254/515] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 298 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:54:23.974008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7976a-3dd8-49a5-9b03-301e8bc70e48 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:24.003173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7c144-991c-4dd9-9865-6373fbe012b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:24.003873 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7c144-991c-4dd9-9865-6373fbe012b2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:24.004189 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7c144-991c-4dd9-9865-6373fbe012b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:24.004476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe7c144-991c-4dd9-9865-6373fbe012b2 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:24.005218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7c144-991c-4dd9-9865-6373fbe012b2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:24.005531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7c144-991c-4dd9-9865-6373fbe012b2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:24.006919 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe7c144-991c-4dd9-9865-6373fbe012b2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:24.019036 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 262/516] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:24.019848 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7c144-991c-4dd9-9865-6373fbe012b2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:24.041270 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:24.074559 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c7976a-3dd8-49a5-9b03-301e8bc70e48 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:24.107634 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f8247-feab-415b-86c7-8c19b719d1cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:24.108467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f8247-feab-415b-86c7-8c19b719d1cf None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:24.108752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f8247-feab-415b-86c7-8c19b719d1cf None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:24.109024 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1f8247-feab-415b-86c7-8c19b719d1cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:24.117120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:24.117471 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe7c144-991c-4dd9-9865-6373fbe012b2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:24.119784 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f8247-feab-415b-86c7-8c19b719d1cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:24.121373 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1f8247-feab-415b-86c7-8c19b719d1cf None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:24.139398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-aa1f8247-feab-415b-86c7-8c19b719d1cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:24.139398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:24.139398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:24.139398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:24.139398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:24.139398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:24.139398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:24.139398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:24.139398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:24.139398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:24.139398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:24.139398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:24.139398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:24.139398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:24.139398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:24.140259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:24.140259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:24.140259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:24.140259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:24.140259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:24.140259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:24.140259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:24.140259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:24.140259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:24.140259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:24.140259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:24.140259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:24.140259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:24.140259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:24.140259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:24.140259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:24.140259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:24.141132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:24.141132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:24.141132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:24.141132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:24.141132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 255/517] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:24.141400 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f8247-feab-415b-86c7-8c19b719d1cf None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:24.151907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81014f-a698-444c-b4be-f9263c072455 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:24.152564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81014f-a698-444c-b4be-f9263c072455 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:24.152852 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81014f-a698-444c-b4be-f9263c072455 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:24.153132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb81014f-a698-444c-b4be-f9263c072455 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:24.222884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81014f-a698-444c-b4be-f9263c072455 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:24.224702 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb81014f-a698-444c-b4be-f9263c072455 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:24.243214 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1f8247-feab-415b-86c7-8c19b719d1cf None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:24.248751 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:24.276198 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a450ab5-2e07-45b6-a5ac-69ecb065c2df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:24.276878 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a450ab5-2e07-45b6-a5ac-69ecb065c2df None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:24.277164 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a450ab5-2e07-45b6-a5ac-69ecb065c2df None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:24.277452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a450ab5-2e07-45b6-a5ac-69ecb065c2df None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:24.278170 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a450ab5-2e07-45b6-a5ac-69ecb065c2df None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:24.279684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a450ab5-2e07-45b6-a5ac-69ecb065c2df None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:24.299894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 256/518] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:54:24 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:54:24.304853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a450ab5-2e07-45b6-a5ac-69ecb065c2df None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:24.345643 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 263/519] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:24 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:54:24.346340 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81014f-a698-444c-b4be-f9263c072455 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:24.368117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:24.430505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a450ab5-2e07-45b6-a5ac-69ecb065c2df None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:24.459940 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb81014f-a698-444c-b4be-f9263c072455 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:24.469112 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4339bb6-478f-484a-bad1-8e013288e46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:24.469939 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339bb6-478f-484a-bad1-8e013288e46c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:24.470225 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339bb6-478f-484a-bad1-8e013288e46c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:24.470705 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4339bb6-478f-484a-bad1-8e013288e46c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:24.482578 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:24.482752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4339bb6-478f-484a-bad1-8e013288e46c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:24.484027 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4339bb6-478f-484a-bad1-8e013288e46c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:24.500147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265de17f-5b7f-4f0b-99d6-e01c94e4c3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:24.500956 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265de17f-5b7f-4f0b-99d6-e01c94e4c3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:24.501249 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265de17f-5b7f-4f0b-99d6-e01c94e4c3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:24.501529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-265de17f-5b7f-4f0b-99d6-e01c94e4c3f5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:24.502194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-e4339bb6-478f-484a-bad1-8e013288e46c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:24.502194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:24.502194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:24.502194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:24.502194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:24.502194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:24.502194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:24.502194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:24.502194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:24.502194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:24.502194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:24.502194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:24.502194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:24.502194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:24.502194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:24.503192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:24.503192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:24.503192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:24.503192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:24.503192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:24.503192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:24.503192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:24.503192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:24.503192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:24.503192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:24.503192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:24.503192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:24.503192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:24.503192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:24.503192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:24.503192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:24.503192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:24.504466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:24.504466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:24.504466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:24.504466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:24.504466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 257/520] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:24.504983 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4339bb6-478f-484a-bad1-8e013288e46c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:24.511189 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265de17f-5b7f-4f0b-99d6-e01c94e4c3f5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:24.512760 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-265de17f-5b7f-4f0b-99d6-e01c94e4c3f5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:24.528597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-265de17f-5b7f-4f0b-99d6-e01c94e4c3f5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:24.528597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:24.528597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:24.528597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:24.528597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:24.528597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:24.528597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:24.528597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:24.528597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:24.528597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:24.528597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:24.528597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:24.528597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:24.528597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:24.528597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:24.529513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:24.529513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:24.529513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:24.529513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:24.529513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:24.529513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:24.529513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:24.529513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:24.529513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:24.529513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:24.529513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:24.529513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:24.529513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:24.529513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:24.529513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:24.529513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:24.529513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:24.530369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:24.530369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:24.530369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:24.530369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:24.530369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:24.530369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:24.530369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:24.530369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:24.532765 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 264/521] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:24.533844 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265de17f-5b7f-4f0b-99d6-e01c94e4c3f5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:24.612433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:24.616083 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4339bb6-478f-484a-bad1-8e013288e46c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:24.645660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-265de17f-5b7f-4f0b-99d6-e01c94e4c3f5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:24.659492 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed99339-c443-4244-8dfe-605f3fa6e3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:24.660208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed99339-c443-4244-8dfe-605f3fa6e3bb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:24.660480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed99339-c443-4244-8dfe-605f3fa6e3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:24.660767 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed99339-c443-4244-8dfe-605f3fa6e3bb None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:24.661509 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed99339-c443-4244-8dfe-605f3fa6e3bb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:24.661822 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed99339-c443-4244-8dfe-605f3fa6e3bb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:24.663224 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed99339-c443-4244-8dfe-605f3fa6e3bb None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:24.679851 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 258/522] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:24.679851 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed99339-c443-4244-8dfe-605f3fa6e3bb None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:24.686987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacf260-4d6c-459b-993a-fd0d39ad75b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:24.688256 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacf260-4d6c-459b-993a-fd0d39ad75b3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:24.688435 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacf260-4d6c-459b-993a-fd0d39ad75b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:24.688786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dacf260-4d6c-459b-993a-fd0d39ad75b3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:24.701211 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dacf260-4d6c-459b-993a-fd0d39ad75b3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:24.702942 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dacf260-4d6c-459b-993a-fd0d39ad75b3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:24.721995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-6dacf260-4d6c-459b-993a-fd0d39ad75b3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:24.721995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:24.721995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:24.721995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:24.721995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:24.721995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:24.721995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:24.721995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:24.721995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:24.721995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:24.721995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:24.721995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:24.721995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:24.721995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:24.721995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:24.723132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:24.723132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:24.723132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:24.723132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:24.723132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:24.723132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:24.723132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:24.723132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:24.723132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:24.723132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:24.723132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:24.723132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:24.723132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:24.723132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:24.723132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:24.723132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:24.723132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:24.724198 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:24.724198 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:24.724198 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:24.724198 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:24.724909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 265/523] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:24.725705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacf260-4d6c-459b-993a-fd0d39ad75b3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:24.758086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:24.788317 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed99339-c443-4244-8dfe-605f3fa6e3bb None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:24.824276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b698f44-c5b8-4038-b8f7-67b3ddc4f4fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:24.825047 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b698f44-c5b8-4038-b8f7-67b3ddc4f4fe None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:24.825479 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b698f44-c5b8-4038-b8f7-67b3ddc4f4fe None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:24.825731 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b698f44-c5b8-4038-b8f7-67b3ddc4f4fe None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:24.826576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b698f44-c5b8-4038-b8f7-67b3ddc4f4fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:24.826952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b698f44-c5b8-4038-b8f7-67b3ddc4f4fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:24.828434 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b698f44-c5b8-4038-b8f7-67b3ddc4f4fe None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:24.831218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dacf260-4d6c-459b-993a-fd0d39ad75b3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:24.841323 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 259/524] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:24.842113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b698f44-c5b8-4038-b8f7-67b3ddc4f4fe None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:24.864296 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:24.872574 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdba50f-8dba-4112-bd8a-752eb31ff313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:24.873306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdba50f-8dba-4112-bd8a-752eb31ff313 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:24.873599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdba50f-8dba-4112-bd8a-752eb31ff313 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:24.873882 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cdba50f-8dba-4112-bd8a-752eb31ff313 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:24.874713 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdba50f-8dba-4112-bd8a-752eb31ff313 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:24.875039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdba50f-8dba-4112-bd8a-752eb31ff313 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:24.876530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cdba50f-8dba-4112-bd8a-752eb31ff313 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:24.896999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 266/525] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:24.898184 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdba50f-8dba-4112-bd8a-752eb31ff313 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:24.955481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b698f44-c5b8-4038-b8f7-67b3ddc4f4fe None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:24.990340 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83e138-4ba5-47b1-a8a5-505f1b71fc32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:24.991155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83e138-4ba5-47b1-a8a5-505f1b71fc32 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:24.991447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83e138-4ba5-47b1-a8a5-505f1b71fc32 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:24.991728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f83e138-4ba5-47b1-a8a5-505f1b71fc32 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:24.993268 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:25.000087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83e138-4ba5-47b1-a8a5-505f1b71fc32 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:25.001576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f83e138-4ba5-47b1-a8a5-505f1b71fc32 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:25.003283 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cdba50f-8dba-4112-bd8a-752eb31ff313 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:25.016367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-9f83e138-4ba5-47b1-a8a5-505f1b71fc32 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 15:54:25.016367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:25.016367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:25.016367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:25.016367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:25.016367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:25.016367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:25.016367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:25.016367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:25.016367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:25.016367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:25.016367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:25.016367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:25.016367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:25.016367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:25.016367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:25.017418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:25.018635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:25.018635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:25.018635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:25.018635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:25.018635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:25.018635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:25.018635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:25.018635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:25.018635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 15:54:25.018635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:25.020581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 260/526] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:24 2022] GET /identity/v3/users/neutron => generated 85 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:25.021505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83e138-4ba5-47b1-a8a5-505f1b71fc32 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:25.043840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee8389-c4df-4b51-9d61-bf30eb2cd80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:25.044527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee8389-c4df-4b51-9d61-bf30eb2cd80c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:25.044843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee8389-c4df-4b51-9d61-bf30eb2cd80c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:25.045126 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fee8389-c4df-4b51-9d61-bf30eb2cd80c None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:25.045853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee8389-c4df-4b51-9d61-bf30eb2cd80c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:25.046162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee8389-c4df-4b51-9d61-bf30eb2cd80c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:25.047545 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fee8389-c4df-4b51-9d61-bf30eb2cd80c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:25.063667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 267/527] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:25.063667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee8389-c4df-4b51-9d61-bf30eb2cd80c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:25.112264 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:25.133569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f83e138-4ba5-47b1-a8a5-505f1b71fc32 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:25.167946 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fee8389-c4df-4b51-9d61-bf30eb2cd80c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:25.171257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4b1d4-c976-4fc7-8c3f-da342a900e8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:25.172172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4b1d4-c976-4fc7-8c3f-da342a900e8e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:25.172461 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4b1d4-c976-4fc7-8c3f-da342a900e8e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:25.172741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f4b1d4-c976-4fc7-8c3f-da342a900e8e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:25.179734 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4b1d4-c976-4fc7-8c3f-da342a900e8e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:25.181328 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f4b1d4-c976-4fc7-8c3f-da342a900e8e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:25.195695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-f5f4b1d4-c976-4fc7-8c3f-da342a900e8e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:25.195695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:25.195695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:25.195695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:25.195695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:25.195695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:25.195695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:25.195695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:25.195695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:25.195695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:25.195695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:25.195695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:25.195695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:25.195695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:25.195695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:25.196596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:25.196596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:25.196596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:25.196596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:25.196596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:25.196596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:25.196596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:25.196596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:25.196596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:25.196596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:25.196596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:25.196596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:25.196596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:25.196596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:25.196596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:25.196596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:25.196596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:25.197445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:25.197445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:25.197445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:25.197445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:25.197445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:25.197445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:25.197445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:25.197445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:25.197445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 261/528] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:25.202284 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4b1d4-c976-4fc7-8c3f-da342a900e8e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:25.205809 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bee16c-17c6-45ee-9cb8-dc54ef83468c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:25.206470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bee16c-17c6-45ee-9cb8-dc54ef83468c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:25.206763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bee16c-17c6-45ee-9cb8-dc54ef83468c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:25.207047 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90bee16c-17c6-45ee-9cb8-dc54ef83468c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:25.207953 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bee16c-17c6-45ee-9cb8-dc54ef83468c None admin] RBAC: Adding query filter params (role.id=83bf7d9c29d4464dab8030f6d706cd6d, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=4751abc9f67947b3ad17ba2b78ce2c12) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:25.208621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bee16c-17c6-45ee-9cb8-dc54ef83468c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:25.210092 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90bee16c-17c6-45ee-9cb8-dc54ef83468c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:25.223851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 268/529] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:54:25 2022] GET /identity/v3/role_assignments?user.id=4751abc9f67947b3ad17ba2b78ce2c12&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=83bf7d9c29d4464dab8030f6d706cd6d => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:25.223851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bee16c-17c6-45ee-9cb8-dc54ef83468c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:25.244888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:25.332189 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f4b1d4-c976-4fc7-8c3f-da342a900e8e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:25.362028 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90bee16c-17c6-45ee-9cb8-dc54ef83468c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:25.381597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2f1e5-904f-406c-8cc1-eaa31eb0605d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:25.382697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2f1e5-904f-406c-8cc1-eaa31eb0605d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:25.383002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2f1e5-904f-406c-8cc1-eaa31eb0605d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:25.383286 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b2f1e5-904f-406c-8cc1-eaa31eb0605d None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:25.384216 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2f1e5-904f-406c-8cc1-eaa31eb0605d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:25.384537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2f1e5-904f-406c-8cc1-eaa31eb0605d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:25.386020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b2f1e5-904f-406c-8cc1-eaa31eb0605d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:25.400454 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:25.403055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456eca74-81e9-455a-835f-b900e576a0cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:25.404247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456eca74-81e9-455a-835f-b900e576a0cf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:25.404742 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456eca74-81e9-455a-835f-b900e576a0cf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:25.405233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-456eca74-81e9-455a-835f-b900e576a0cf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:25.415238 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 262/530] 10.222.0.85 () {56 vars in 1139 bytes} [Wed May 25 15:54:25 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:25.415238 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2f1e5-904f-406c-8cc1-eaa31eb0605d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:25.418100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456eca74-81e9-455a-835f-b900e576a0cf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:25.421315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-456eca74-81e9-455a-835f-b900e576a0cf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:25.443691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-456eca74-81e9-455a-835f-b900e576a0cf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:25.443691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:25.443691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:25.443691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:25.443691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:25.443691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:25.443691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:25.443691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:25.443691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:25.443691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:25.443691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:25.443691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:25.443691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:25.443691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:25.443691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:25.444871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:25.444871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:25.444871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:25.444871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:25.444871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:25.444871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:25.444871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:25.444871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:25.444871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:25.444871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:25.444871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:25.444871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:25.444871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:25.444871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:25.444871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:25.444871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:25.444871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:25.445950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:25.445950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:25.445950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:25.445950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:25.445950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:25.445950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:25.445950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:25.445950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:25.448005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 269/531] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:25.452833 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456eca74-81e9-455a-835f-b900e576a0cf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:25.547533 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b2f1e5-904f-406c-8cc1-eaa31eb0605d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:25.552795 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:25.584437 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-456eca74-81e9-455a-835f-b900e576a0cf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:25.591066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a20c01-14ae-4472-bc8f-18aa5101f48b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:25.592036 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a20c01-14ae-4472-bc8f-18aa5101f48b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:25.592492 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a20c01-14ae-4472-bc8f-18aa5101f48b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:25.592931 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a20c01-14ae-4472-bc8f-18aa5101f48b None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:25.594298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a20c01-14ae-4472-bc8f-18aa5101f48b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:25.594770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a20c01-14ae-4472-bc8f-18aa5101f48b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:25.596765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a20c01-14ae-4472-bc8f-18aa5101f48b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:25.623870 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab978a-0972-4de1-82fb-ce42b1480bbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:25.624113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab978a-0972-4de1-82fb-ce42b1480bbb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:25.624464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab978a-0972-4de1-82fb-ce42b1480bbb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:25.624835 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab978a-0972-4de1-82fb-ce42b1480bbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:25.625698 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 263/532] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:25.626609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a20c01-14ae-4472-bc8f-18aa5101f48b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:25.691272 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:25.706410 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab978a-0972-4de1-82fb-ce42b1480bbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:25.707983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28ab978a-0972-4de1-82fb-ce42b1480bbb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:25.736420 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a20c01-14ae-4472-bc8f-18aa5101f48b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:25.771994 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2795b-090a-4164-a5fa-ff57abc4724b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:25.773022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2795b-090a-4164-a5fa-ff57abc4724b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:25.773497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2795b-090a-4164-a5fa-ff57abc4724b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:25.773984 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d2795b-090a-4164-a5fa-ff57abc4724b None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:25.775046 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2795b-090a-4164-a5fa-ff57abc4724b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:25.775593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2795b-090a-4164-a5fa-ff57abc4724b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:25.777292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d2795b-090a-4164-a5fa-ff57abc4724b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:25.786165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:25.796022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 264/533] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:25.797185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2795b-090a-4164-a5fa-ff57abc4724b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:25.815262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 270/534] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:54:25.815262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab978a-0972-4de1-82fb-ce42b1480bbb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:25.906432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d2795b-090a-4164-a5fa-ff57abc4724b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:25.911997 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab978a-0972-4de1-82fb-ce42b1480bbb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:25.928377 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:25.947302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1d024-928a-4256-8a57-e93ce87fbfbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:25.948075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1d024-928a-4256-8a57-e93ce87fbfbe None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:25.948288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1d024-928a-4256-8a57-e93ce87fbfbe None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:25.948572 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa1d024-928a-4256-8a57-e93ce87fbfbe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:25.949409 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1d024-928a-4256-8a57-e93ce87fbfbe None admin] RBAC: Adding query filter params (role.id=83bf7d9c29d4464dab8030f6d706cd6d, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=196dc406a95444c39babc50d59f9eb6b) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:25.949733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1d024-928a-4256-8a57-e93ce87fbfbe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:25.951269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aa1d024-928a-4256-8a57-e93ce87fbfbe None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:25.953520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3c247-0f93-4bba-b87f-a5ad2fe22537 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:25.954885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3c247-0f93-4bba-b87f-a5ad2fe22537 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:25.955608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3c247-0f93-4bba-b87f-a5ad2fe22537 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:25.956094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e3c247-0f93-4bba-b87f-a5ad2fe22537 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:25.957259 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 271/535] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:54:25 2022] GET /identity/v3/role_assignments?user.id=196dc406a95444c39babc50d59f9eb6b&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=83bf7d9c29d4464dab8030f6d706cd6d => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:25.957627 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3c247-0f93-4bba-b87f-a5ad2fe22537 None admin] RBAC: Adding query filter params (role.id=83bf7d9c29d4464dab8030f6d706cd6d, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=f81aecdc87d84501ba94b208405b51dd) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:25.957868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1d024-928a-4256-8a57-e93ce87fbfbe None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:25.958680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3c247-0f93-4bba-b87f-a5ad2fe22537 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:25.960439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e3c247-0f93-4bba-b87f-a5ad2fe22537 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:25.969642 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 265/536] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:54:25 2022] GET /identity/v3/role_assignments?user.id=f81aecdc87d84501ba94b208405b51dd&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=83bf7d9c29d4464dab8030f6d706cd6d => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:26.008206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:26.076553 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa1d024-928a-4256-8a57-e93ce87fbfbe None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:26.116086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc388b-a6a7-4453-9ca8-2d1c009cf317 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:26.116917 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc388b-a6a7-4453-9ca8-2d1c009cf317 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:26.117216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc388b-a6a7-4453-9ca8-2d1c009cf317 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:26.117498 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33dc388b-a6a7-4453-9ca8-2d1c009cf317 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:26.130168 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc388b-a6a7-4453-9ca8-2d1c009cf317 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:26.131701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33dc388b-a6a7-4453-9ca8-2d1c009cf317 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:26.151292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-33dc388b-a6a7-4453-9ca8-2d1c009cf317 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:26.151292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:26.151292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:26.151292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:26.151292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:26.151292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:26.151292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:26.151292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:26.151292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:26.151292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:26.151292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:26.151292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:26.151292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:26.151292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:26.151292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:26.152369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:26.152369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:26.152369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:26.152369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:26.152369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:26.152369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:26.152369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:26.152369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:26.152369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:26.152369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:26.152369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:26.152369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:26.152369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:26.152369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:26.152369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:26.152369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:26.152369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:26.153213 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:26.153213 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:26.153213 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:26.153213 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:26.153213 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 272/537] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:26.161517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3c247-0f93-4bba-b87f-a5ad2fe22537 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:26.299372 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e3c247-0f93-4bba-b87f-a5ad2fe22537 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:26.334284 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfb28c-a7cd-4ded-99c6-910b6d66fd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:26.334957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfb28c-a7cd-4ded-99c6-910b6d66fd99 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:26.335241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfb28c-a7cd-4ded-99c6-910b6d66fd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:26.335523 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ecfb28c-a7cd-4ded-99c6-910b6d66fd99 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:26.336282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfb28c-a7cd-4ded-99c6-910b6d66fd99 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:26.336597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfb28c-a7cd-4ded-99c6-910b6d66fd99 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:26.338157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ecfb28c-a7cd-4ded-99c6-910b6d66fd99 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:26.350833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 266/538] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:26.356249 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc388b-a6a7-4453-9ca8-2d1c009cf317 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:26.500679 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:26.509920 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33dc388b-a6a7-4453-9ca8-2d1c009cf317 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:26.555776 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0567d448-fa77-4e29-895f-883cbde2272d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:26.556629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567d448-fa77-4e29-895f-883cbde2272d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:26.556916 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567d448-fa77-4e29-895f-883cbde2272d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:26.557205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0567d448-fa77-4e29-895f-883cbde2272d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:26.564397 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567d448-fa77-4e29-895f-883cbde2272d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:26.565950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0567d448-fa77-4e29-895f-883cbde2272d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:26.580383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0567d448-fa77-4e29-895f-883cbde2272d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:26.580383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:26.580383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:26.580383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:26.580383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:26.580383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:26.580383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:26.580383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:26.580383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:26.580383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:26.580383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:26.580383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:26.580383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:26.580383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:26.580383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:26.581262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:26.581262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:26.581262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:26.581262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:26.581262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:26.581262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:26.581262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:26.581262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:26.581262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:26.581262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:26.581262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:26.581262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:26.581262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:26.581262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:26.581262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:26.581262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:26.581262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:26.582125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:26.582125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:26.582125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:26.582125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:26.582125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:26.582125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:26.582125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:26.582125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:26.582125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 273/539] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:26.587638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfb28c-a7cd-4ded-99c6-910b6d66fd99 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:26.662477 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:26.730168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ecfb28c-a7cd-4ded-99c6-910b6d66fd99 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:26.735592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67010ed9-0385-4b48-8410-87434ecd4d42 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:26.735916 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67010ed9-0385-4b48-8410-87434ecd4d42 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:26.736205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67010ed9-0385-4b48-8410-87434ecd4d42 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:26.740283 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 274/540] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:26.745442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66512adf-ad19-4bd6-8f65-040f193af259 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:26.745745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66512adf-ad19-4bd6-8f65-040f193af259 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:26.746035 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66512adf-ad19-4bd6-8f65-040f193af259 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:26.772906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-66512adf-ad19-4bd6-8f65-040f193af259 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:26.774024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-66512adf-ad19-4bd6-8f65-040f193af259 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:26.786925 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32a1a5-a35f-48cf-9cb2-b16f5e97dada None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:26.787603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32a1a5-a35f-48cf-9cb2-b16f5e97dada None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:26.787911 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32a1a5-a35f-48cf-9cb2-b16f5e97dada None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:26.788191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32a1a5-a35f-48cf-9cb2-b16f5e97dada None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:26.788990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32a1a5-a35f-48cf-9cb2-b16f5e97dada None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:26.789308 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32a1a5-a35f-48cf-9cb2-b16f5e97dada None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:26.790767 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32a1a5-a35f-48cf-9cb2-b16f5e97dada None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:26.805754 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 267/541] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:26.811106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32a1a5-a35f-48cf-9cb2-b16f5e97dada None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:26.878174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 275/542] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:54:26.930503 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66512adf-ad19-4bd6-8f65-040f193af259 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:26.955853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32a1a5-a35f-48cf-9cb2-b16f5e97dada None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:26.957614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:26.996761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33ed95-5d33-4440-9b4b-b6dfca21f0b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:26.997438 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33ed95-5d33-4440-9b4b-b6dfca21f0b3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:26.997772 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33ed95-5d33-4440-9b4b-b6dfca21f0b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:26.998061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f33ed95-5d33-4440-9b4b-b6dfca21f0b3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:26.998922 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33ed95-5d33-4440-9b4b-b6dfca21f0b3 None admin] RBAC: Adding query filter params (role.id=83bf7d9c29d4464dab8030f6d706cd6d, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=413b3d65427b41dc99bc7b0a94fb1dc8) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:26.999246 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33ed95-5d33-4440-9b4b-b6dfca21f0b3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:27.000898 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f33ed95-5d33-4440-9b4b-b6dfca21f0b3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:27.006981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 268/543] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:54:26 2022] GET /identity/v3/role_assignments?user.id=413b3d65427b41dc99bc7b0a94fb1dc8&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=83bf7d9c29d4464dab8030f6d706cd6d => generated 259 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:27.095553 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:27.123603 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66512adf-ad19-4bd6-8f65-040f193af259 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:27.174116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81110cb-b629-4a5e-80c1-31175c80714e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:27.174943 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81110cb-b629-4a5e-80c1-31175c80714e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:27.175229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81110cb-b629-4a5e-80c1-31175c80714e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:27.175508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81110cb-b629-4a5e-80c1-31175c80714e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:27.192793 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81110cb-b629-4a5e-80c1-31175c80714e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:27.200565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a81110cb-b629-4a5e-80c1-31175c80714e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:27.200565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:27.200565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:27.200565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:27.200565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:27.200565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:27.200565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:27.200565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:27.200565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:27.200565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:27.200565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:27.200565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:27.200565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:27.200565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:27.200565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:27.200565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:27.201492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:27.201492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:27.201492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:27.201492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:27.201492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:27.201492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:27.201492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:27.201492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:27.201492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:27.201492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:27.201492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:27.201492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:27.201492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:27.201492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:27.201492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:27.201492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:27.202294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:27.202294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:27.202294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:27.202294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:27.202294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:27.202294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:27.202294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:27.202294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:27.202294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:27.202294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:27.202294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:27.202294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:27.202294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 276/544] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:26 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:27.209738 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33ed95-5d33-4440-9b4b-b6dfca21f0b3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:27.350699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f33ed95-5d33-4440-9b4b-b6dfca21f0b3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:27.388282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b83a7d-2f45-47c7-a4af-e98e488b24e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:27.388980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b83a7d-2f45-47c7-a4af-e98e488b24e6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:27.389278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b83a7d-2f45-47c7-a4af-e98e488b24e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:27.389558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b83a7d-2f45-47c7-a4af-e98e488b24e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:27.390328 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b83a7d-2f45-47c7-a4af-e98e488b24e6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:27.390642 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b83a7d-2f45-47c7-a4af-e98e488b24e6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:27.392080 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b83a7d-2f45-47c7-a4af-e98e488b24e6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:27.401168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 269/545] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:27.406588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81110cb-b629-4a5e-80c1-31175c80714e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:27.551844 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81110cb-b629-4a5e-80c1-31175c80714e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:27.598827 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d08084b-83c7-4859-b679-507c4449b352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:27.599660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08084b-83c7-4859-b679-507c4449b352 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:27.599979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08084b-83c7-4859-b679-507c4449b352 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:27.600258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d08084b-83c7-4859-b679-507c4449b352 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:27.617555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d08084b-83c7-4859-b679-507c4449b352 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:27.619036 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d08084b-83c7-4859-b679-507c4449b352 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:27.620920 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:27.642705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4d08084b-83c7-4859-b679-507c4449b352 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:27.642705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:27.642705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:27.642705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:27.642705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:27.642705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:27.642705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:27.642705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:27.642705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:27.642705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:27.642705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:27.642705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:27.642705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:27.642705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:27.642705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:27.643645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:27.643645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:27.643645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:27.643645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:27.643645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:27.643645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:27.643645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:27.643645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:27.643645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:27.643645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:27.643645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:27.643645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:27.643645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:27.643645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:27.643645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:27.643645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:27.643645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:27.644527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:27.644527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:27.644527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:27.644527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:27.644527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 277/546] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:27.650738 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b83a7d-2f45-47c7-a4af-e98e488b24e6 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:27.709102 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:27.781183 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244ab6a-8eb9-4199-a267-6956fafede62 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:27.781489 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244ab6a-8eb9-4199-a267-6956fafede62 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:27.781746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a244ab6a-8eb9-4199-a267-6956fafede62 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:27.783366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 278/547] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:27.788725 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a2212-2439-40a0-9578-63bea8bce694 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:27.789311 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a2212-2439-40a0-9578-63bea8bce694 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:27.789791 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-587a2212-2439-40a0-9578-63bea8bce694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:27.807159 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-587a2212-2439-40a0-9578-63bea8bce694 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:27.808674 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-587a2212-2439-40a0-9578-63bea8bce694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:27.824691 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b83a7d-2f45-47c7-a4af-e98e488b24e6 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:27.878040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 279/548] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:54:27.901448 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a2212-2439-40a0-9578-63bea8bce694 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:27.912987 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea13230-d023-46a9-98f3-c6855b0db441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:27.913990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea13230-d023-46a9-98f3-c6855b0db441 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:27.914478 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea13230-d023-46a9-98f3-c6855b0db441 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:27.914956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea13230-d023-46a9-98f3-c6855b0db441 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:27.915969 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea13230-d023-46a9-98f3-c6855b0db441 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:27.916483 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea13230-d023-46a9-98f3-c6855b0db441 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:27.918095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea13230-d023-46a9-98f3-c6855b0db441 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:27.944087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 270/549] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:27.949261 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea13230-d023-46a9-98f3-c6855b0db441 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:28.014633 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-587a2212-2439-40a0-9578-63bea8bce694 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:28.062467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:28.084252 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:28.093636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18e356-9ef0-4c46-ae1b-a91d67b60f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:28.094856 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18e356-9ef0-4c46-ae1b-a91d67b60f1e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:28.094856 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18e356-9ef0-4c46-ae1b-a91d67b60f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:28.095064 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b18e356-9ef0-4c46-ae1b-a91d67b60f1e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:28.106751 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea13230-d023-46a9-98f3-c6855b0db441 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:28.124315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b18e356-9ef0-4c46-ae1b-a91d67b60f1e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:28.139130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3b18e356-9ef0-4c46-ae1b-a91d67b60f1e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:28.139130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:28.139130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:28.139130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:28.139130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:28.139130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:28.139130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:28.139130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:28.139130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:28.139130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:28.139130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:28.139130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:28.139130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:28.139130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:28.139130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:28.139130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:28.140750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:28.140750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:28.140750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:28.140750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:28.140750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:28.140750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:28.140750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:28.140750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:28.140750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:28.140750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:28.140750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:28.140750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:28.140750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:28.140750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:28.140750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:28.140750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:28.141748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:28.141748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:28.141748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:28.141748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:28.141748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:28.141748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:28.141748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:28.141748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:28.141748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:28.141748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:28.141748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:28.141748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:28.144289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 280/550] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 243 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:28.148957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18e356-9ef0-4c46-ae1b-a91d67b60f1e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:28.164541 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf02e5f-fa77-4b62-bfad-27debe210b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:28.165371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf02e5f-fa77-4b62-bfad-27debe210b80 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:28.165661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf02e5f-fa77-4b62-bfad-27debe210b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:28.165941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf02e5f-fa77-4b62-bfad-27debe210b80 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:28.179963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf02e5f-fa77-4b62-bfad-27debe210b80 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:28.179963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf02e5f-fa77-4b62-bfad-27debe210b80 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:28.195861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-9bf02e5f-fa77-4b62-bfad-27debe210b80 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 15:54:28.195861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:28.195861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:28.195861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:28.195861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:28.195861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:28.195861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:28.195861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:28.195861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:28.195861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:28.195861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:28.195861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:28.195861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:28.195861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:28.195861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:28.195861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:28.197118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:28.198230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:28.198230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:28.198230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:28.198230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:28.198230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:28.198230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:28.198230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:28.198230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:28.198230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 15:54:28.198230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:28.198230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 271/551] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:54:27 2022] GET /identity/v3/users/swift => generated 83 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:54:28.205023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf02e5f-fa77-4b62-bfad-27debe210b80 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:28.326091 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b18e356-9ef0-4c46-ae1b-a91d67b60f1e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:28.364878 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6a246-dcaf-457e-9c40-f27aee6afda5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:28.365921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6a246-dcaf-457e-9c40-f27aee6afda5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:28.366429 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6a246-dcaf-457e-9c40-f27aee6afda5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:28.366949 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc6a246-dcaf-457e-9c40-f27aee6afda5 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:28.367949 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6a246-dcaf-457e-9c40-f27aee6afda5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:28.368795 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6a246-dcaf-457e-9c40-f27aee6afda5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:28.370433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc6a246-dcaf-457e-9c40-f27aee6afda5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:28.380950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 281/552] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:28.389387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6a246-dcaf-457e-9c40-f27aee6afda5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:28.449642 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:28.461968 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf02e5f-fa77-4b62-bfad-27debe210b80 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:28.518968 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc6a246-dcaf-457e-9c40-f27aee6afda5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:28.557219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9726a-c9f0-4abb-8d29-a32ad50cb659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:28.558395 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9726a-c9f0-4abb-8d29-a32ad50cb659 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:28.558887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9726a-c9f0-4abb-8d29-a32ad50cb659 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:28.559396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9726a-c9f0-4abb-8d29-a32ad50cb659 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:28.561603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e87f91-290c-445b-ac12-33112be7cdc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:28.563732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e87f91-290c-445b-ac12-33112be7cdc8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:28.564976 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e87f91-290c-445b-ac12-33112be7cdc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:28.569575 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04e87f91-290c-445b-ac12-33112be7cdc8 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:28.570680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e87f91-290c-445b-ac12-33112be7cdc8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:28.571208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e87f91-290c-445b-ac12-33112be7cdc8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:28.572916 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04e87f91-290c-445b-ac12-33112be7cdc8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:28.585468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9726a-c9f0-4abb-8d29-a32ad50cb659 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:28.587269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9726a-c9f0-4abb-8d29-a32ad50cb659 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:28.613601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 272/553] 10.222.0.85 () {56 vars in 1135 bytes} [Wed May 25 15:54:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 409 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:28.616531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a9a9726a-c9f0-4abb-8d29-a32ad50cb659 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:28.616531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:28.616531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:28.616531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:28.616531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:28.616531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:28.616531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:28.616531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:28.616531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:28.616531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:28.616531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:28.616531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:28.616531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:28.616531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:28.616531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:28.617762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:28.617762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:28.617762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:28.617762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:28.617762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:28.617762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:28.617762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:28.617762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:28.617762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:28.617762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:28.617762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:28.617762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:28.617762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:28.617762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:28.617762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:28.617762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:28.617762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:28.618687 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:28.618687 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:28.618687 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:28.618687 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:28.618907 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e87f91-290c-445b-ac12-33112be7cdc8 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:28.621110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 282/554] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:28.626700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9726a-c9f0-4abb-8d29-a32ad50cb659 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:28.632963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:28.755610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:28.759792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04e87f91-290c-445b-ac12-33112be7cdc8 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:28.761135 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9726a-c9f0-4abb-8d29-a32ad50cb659 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:28.796094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e3706be-3c08-44c0-90db-8c251ddb404e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:28.796627 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3706be-3c08-44c0-90db-8c251ddb404e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:28.796914 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3706be-3c08-44c0-90db-8c251ddb404e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:28.797192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3706be-3c08-44c0-90db-8c251ddb404e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:28.805352 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-387b9fc6-b90c-4d3c-84c2-f350f46eaab5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:28.806282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b9fc6-b90c-4d3c-84c2-f350f46eaab5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:28.807171 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b9fc6-b90c-4d3c-84c2-f350f46eaab5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:28.807637 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387b9fc6-b90c-4d3c-84c2-f350f46eaab5 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:28.808646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b9fc6-b90c-4d3c-84c2-f350f46eaab5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:28.809467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b9fc6-b90c-4d3c-84c2-f350f46eaab5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:28.811722 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-387b9fc6-b90c-4d3c-84c2-f350f46eaab5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:28.827996 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 283/555] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:28.830436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bd8de-bd81-4b22-9a97-2c9cdefe6c98 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:28.830963 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bd8de-bd81-4b22-9a97-2c9cdefe6c98 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:28.831450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883bd8de-bd81-4b22-9a97-2c9cdefe6c98 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:28.834511 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 284/556] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:28 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:28.835297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bd8de-bd81-4b22-9a97-2c9cdefe6c98 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:28.870371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3706be-3c08-44c0-90db-8c251ddb404e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:28.871883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e3706be-3c08-44c0-90db-8c251ddb404e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:28.880311 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:28.949366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883bd8de-bd81-4b22-9a97-2c9cdefe6c98 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:28.980687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 273/557] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 15:54:28.980687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597103bd-8b35-4b2f-a741-e82b887b0c87 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:28.980687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597103bd-8b35-4b2f-a741-e82b887b0c87 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:28.980687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-597103bd-8b35-4b2f-a741-e82b887b0c87 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:28.980687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 274/558] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:28.981121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a00b3-d32a-4421-bf25-d107ceeef084 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:28.981121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a00b3-d32a-4421-bf25-d107ceeef084 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:28.981761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38a00b3-d32a-4421-bf25-d107ceeef084 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:28.993327 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894754f2-8a7e-4c2c-a345-f09b8a2c51d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:28.994217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894754f2-8a7e-4c2c-a345-f09b8a2c51d3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:28.994506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894754f2-8a7e-4c2c-a345-f09b8a2c51d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:28.994782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894754f2-8a7e-4c2c-a345-f09b8a2c51d3 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:28.999861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d38a00b3-d32a-4421-bf25-d107ceeef084 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:29.001079 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d38a00b3-d32a-4421-bf25-d107ceeef084 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:29.002446 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894754f2-8a7e-4c2c-a345-f09b8a2c51d3 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:29.003959 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894754f2-8a7e-4c2c-a345-f09b8a2c51d3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:29.018876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-894754f2-8a7e-4c2c-a345-f09b8a2c51d3 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 15:54:29.018876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:29.018876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:29.018876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:29.018876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:29.018876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:29.018876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:29.018876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:29.018876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:29.018876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:29.018876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:29.018876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:29.018876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:29.018876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:29.018876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:29.018876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:29.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:29.020927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:29.020927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:29.020927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:29.020927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:29.020927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:29.020927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:29.020927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:29.020927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:29.020927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 15:54:29.020927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:29.020927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 285/559] 10.222.0.85 () {56 vars in 1093 bytes} [Wed May 25 15:54:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:29.021533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894754f2-8a7e-4c2c-a345-f09b8a2c51d3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:29.041378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:29.079666 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 275/560] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:29.080880 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8110fa-b550-4272-94a0-6d19edd638bc None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:29.081179 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8110fa-b550-4272-94a0-6d19edd638bc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:29.081465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8110fa-b550-4272-94a0-6d19edd638bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:29.104573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-cd8110fa-b550-4272-94a0-6d19edd638bc None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:29.105987 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-cd8110fa-b550-4272-94a0-6d19edd638bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:29.138365 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:29.142915 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894754f2-8a7e-4c2c-a345-f09b8a2c51d3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:29.183881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 276/561] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:29.184082 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ebfc4-425f-49fa-abcd-2c0a8c021c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:29.184194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8110fa-b550-4272-94a0-6d19edd638bc None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:29.184282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ebfc4-425f-49fa-abcd-2c0a8c021c02 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:29.184282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ebfc4-425f-49fa-abcd-2c0a8c021c02 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:29.184282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d02ebfc4-425f-49fa-abcd-2c0a8c021c02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:29.201506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ebfc4-425f-49fa-abcd-2c0a8c021c02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:29.203087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d02ebfc4-425f-49fa-abcd-2c0a8c021c02 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:29.221826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d02ebfc4-425f-49fa-abcd-2c0a8c021c02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:29.221826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:29.221826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:29.221826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:29.221826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:29.221826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:29.221826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:29.221826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:29.221826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:29.221826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:29.221826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:29.221826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:29.221826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:29.221826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:29.221826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:29.222988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:29.222988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:29.222988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:29.222988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:29.222988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:29.222988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:29.222988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:29.222988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:29.222988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:29.222988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:29.222988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:29.222988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:29.222988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:29.222988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:29.222988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:29.222988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:29.222988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:29.224217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:29.224217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:29.224217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:29.224217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:29.228236 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 286/562] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:29.228236 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ebfc4-425f-49fa-abcd-2c0a8c021c02 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:29.296266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:29.305652 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8110fa-b550-4272-94a0-6d19edd638bc None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:29.329582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d02ebfc4-425f-49fa-abcd-2c0a8c021c02 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:29.346582 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1346e-8fed-40b9-82c0-7ee1c9630f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:29.347531 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1346e-8fed-40b9-82c0-7ee1c9630f77 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:29.347955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1346e-8fed-40b9-82c0-7ee1c9630f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:29.348452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa1346e-8fed-40b9-82c0-7ee1c9630f77 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:29.349950 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1346e-8fed-40b9-82c0-7ee1c9630f77 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:29.350273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1346e-8fed-40b9-82c0-7ee1c9630f77 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:29.351772 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa1346e-8fed-40b9-82c0-7ee1c9630f77 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:29.367267 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9530c-bed1-4b03-8f1c-2d2f0c61f44b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:29.368113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9530c-bed1-4b03-8f1c-2d2f0c61f44b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:29.368385 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9530c-bed1-4b03-8f1c-2d2f0c61f44b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:29.368681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b9530c-bed1-4b03-8f1c-2d2f0c61f44b None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:29.370792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 277/563] 10.222.0.85 () {56 vars in 1133 bytes} [Wed May 25 15:54:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:29.371499 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1346e-8fed-40b9-82c0-7ee1c9630f77 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:29.384189 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b9530c-bed1-4b03-8f1c-2d2f0c61f44b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:29.392147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-33b9530c-bed1-4b03-8f1c-2d2f0c61f44b None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:29.392147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:29.392147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:29.392147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:29.392147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:29.392147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:29.392147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:29.392147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:29.392147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:29.392147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:29.392147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:29.392147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:29.392147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:29.392147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:29.392147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:29.392147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:29.393099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:29.393099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:29.393099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:29.393099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:29.393099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:29.393099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:29.393099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:29.393099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:29.393099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:29.393099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:29.393099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:29.393099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:29.393099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:29.393099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:29.393099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:29.393099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:29.393947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:29.393947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:29.393947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:29.393947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:29.393947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:29.393947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:29.393947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:29.393947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:29.393947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:29.393947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:29.393947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:29.393947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:29.393947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 287/564] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:29.394649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9530c-bed1-4b03-8f1c-2d2f0c61f44b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:29.401528 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:29.476998 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa1346e-8fed-40b9-82c0-7ee1c9630f77 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:29.504295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b9530c-bed1-4b03-8f1c-2d2f0c61f44b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:29.510831 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4caed3-07c7-486c-97f1-70713e63b357 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:29.511650 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4caed3-07c7-486c-97f1-70713e63b357 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:29.511969 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4caed3-07c7-486c-97f1-70713e63b357 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:29.512254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4caed3-07c7-486c-97f1-70713e63b357 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:29.522942 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4caed3-07c7-486c-97f1-70713e63b357 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:29.531002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-6e4caed3-07c7-486c-97f1-70713e63b357 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:29.531002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:29.531002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:29.531002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:29.531002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:29.531002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:29.531002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:29.531002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:29.531002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:29.531002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:29.531002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:29.531002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:29.531002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:29.531002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:29.531002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:29.531002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:29.532017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:29.532017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:29.532017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:29.532017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:29.532017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:29.532017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:29.532017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:29.532017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:29.532017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:29.532017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:29.532017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:29.532017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:29.532017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:29.532017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:29.532017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:29.532017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:29.532881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:29.532881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:29.532881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:29.532881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:29.532881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:29.532881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:29.532881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:29.532881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:29.532881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:29.532881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:29.532881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:29.532881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:29.532881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 278/565] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:29 2022] GET /identity/v3/roles/service => generated 85 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:29.533584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:29.533657 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4caed3-07c7-486c-97f1-70713e63b357 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:29.541973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e7a99-957a-4433-a8c5-f933f51a2a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:29.542645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e7a99-957a-4433-a8c5-f933f51a2a53 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:29.542931 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e7a99-957a-4433-a8c5-f933f51a2a53 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:29.543208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3e7a99-957a-4433-a8c5-f933f51a2a53 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:29.543975 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e7a99-957a-4433-a8c5-f933f51a2a53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:29.544336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e7a99-957a-4433-a8c5-f933f51a2a53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:29.545716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee3e7a99-957a-4433-a8c5-f933f51a2a53 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:29.557529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 288/566] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:29.558240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e7a99-957a-4433-a8c5-f933f51a2a53 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:29.616416 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:29.634112 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4caed3-07c7-486c-97f1-70713e63b357 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:29.657407 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3e7a99-957a-4433-a8c5-f933f51a2a53 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:29.666463 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0946f4b-b829-4573-8076-c43360ea9fa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:29.667258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0946f4b-b829-4573-8076-c43360ea9fa1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:29.667542 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0946f4b-b829-4573-8076-c43360ea9fa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:29.667836 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0946f4b-b829-4573-8076-c43360ea9fa1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:29.678071 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0946f4b-b829-4573-8076-c43360ea9fa1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:29.679540 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0946f4b-b829-4573-8076-c43360ea9fa1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:29.690450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d48ab3-5feb-48a3-ab87-f99d2ba4121e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:29.691125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d48ab3-5feb-48a3-ab87-f99d2ba4121e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:29.691416 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d48ab3-5feb-48a3-ab87-f99d2ba4121e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:29.691700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d48ab3-5feb-48a3-ab87-f99d2ba4121e None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:29.692502 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d48ab3-5feb-48a3-ab87-f99d2ba4121e None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:29.692822 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d48ab3-5feb-48a3-ab87-f99d2ba4121e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:29.694195 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5d48ab3-5feb-48a3-ab87-f99d2ba4121e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:29.696142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-d0946f4b-b829-4573-8076-c43360ea9fa1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:29.696142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:29.696142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:29.696142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:29.696142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:29.696142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:29.696142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:29.696142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:29.696142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:29.696142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:29.696142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:29.696142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:29.696142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:29.696142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:29.696142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:29.697166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:29.697166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:29.697166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:29.697166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:29.697166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:29.697166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:29.697166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:29.697166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:29.697166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:29.697166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:29.697166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:29.697166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:29.697166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:29.697166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:29.697166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:29.697166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:29.697166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:29.698137 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:29.698137 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:29.698137 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:29.698137 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:29.698137 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 279/567] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:29.698840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666978f-adf0-489a-8eef-5b7931c19621 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:29.699124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666978f-adf0-489a-8eef-5b7931c19621 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:29.699398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3666978f-adf0-489a-8eef-5b7931c19621 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:29.703937 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 280/568] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:29.703937 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666978f-adf0-489a-8eef-5b7931c19621 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:29.706752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 289/569] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:29.707568 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d48ab3-5feb-48a3-ab87-f99d2ba4121e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:29.760695 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:29.806956 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d48ab3-5feb-48a3-ab87-f99d2ba4121e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:29.807275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3666978f-adf0-489a-8eef-5b7931c19621 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:29.841264 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d49908-dcd5-4268-ac65-8e2c66b7a3c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:29.841866 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d49908-dcd5-4268-ac65-8e2c66b7a3c1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:29.842157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d49908-dcd5-4268-ac65-8e2c66b7a3c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:29.842442 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97d49908-dcd5-4268-ac65-8e2c66b7a3c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:29.842948 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17964df-1f04-419c-9ad5-0bf5a3cb3dd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:29.843194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d49908-dcd5-4268-ac65-8e2c66b7a3c1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:29.843509 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d49908-dcd5-4268-ac65-8e2c66b7a3c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:29.843748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17964df-1f04-419c-9ad5-0bf5a3cb3dd2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:29.844063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17964df-1f04-419c-9ad5-0bf5a3cb3dd2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:29.844346 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17964df-1f04-419c-9ad5-0bf5a3cb3dd2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:29.844932 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97d49908-dcd5-4268-ac65-8e2c66b7a3c1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:29.847240 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:29.851165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17964df-1f04-419c-9ad5-0bf5a3cb3dd2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:29.852639 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c17964df-1f04-419c-9ad5-0bf5a3cb3dd2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:29.853709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 281/570] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:29 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:29.854676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d49908-dcd5-4268-ac65-8e2c66b7a3c1 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:29.867671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c17964df-1f04-419c-9ad5-0bf5a3cb3dd2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:29.867671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:29.867671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:29.867671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:29.867671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:29.867671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:29.867671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:29.867671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:29.867671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:29.867671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:29.867671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:29.867671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:29.867671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:29.867671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:29.867671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:29.868629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:29.868629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:29.868629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:29.868629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:29.868629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:29.868629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:29.868629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:29.868629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:29.868629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:29.868629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:29.868629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:29.868629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:29.868629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:29.868629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:29.868629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:29.868629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:29.868629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:29.869552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:29.869552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:29.869552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:29.869552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:29.869552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:29.869552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:29.869552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:29.869552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:29.869552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 290/571] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:29.870305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e72ad5-1492-4d08-be7f-22dd9b44e2e2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:29.870595 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e72ad5-1492-4d08-be7f-22dd9b44e2e2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:29.870878 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91e72ad5-1492-4d08-be7f-22dd9b44e2e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:29.886484 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-91e72ad5-1492-4d08-be7f-22dd9b44e2e2 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:29.887675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-91e72ad5-1492-4d08-be7f-22dd9b44e2e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:29.952671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 291/572] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:29.953497 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e72ad5-1492-4d08-be7f-22dd9b44e2e2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:29.961663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97d49908-dcd5-4268-ac65-8e2c66b7a3c1 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:29.996016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015bdd9-e5f1-4d32-a446-01ed30723bed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:29.996684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015bdd9-e5f1-4d32-a446-01ed30723bed None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:29.996969 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015bdd9-e5f1-4d32-a446-01ed30723bed None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:29.997239 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d015bdd9-e5f1-4d32-a446-01ed30723bed None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:29.997989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015bdd9-e5f1-4d32-a446-01ed30723bed None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:29.998295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015bdd9-e5f1-4d32-a446-01ed30723bed None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:29.999657 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d015bdd9-e5f1-4d32-a446-01ed30723bed None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:30.006080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:30.011396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 282/573] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:30.012208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015bdd9-e5f1-4d32-a446-01ed30723bed None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:30.063353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91e72ad5-1492-4d08-be7f-22dd9b44e2e2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:30.069398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:30.096111 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e2886-6bcb-458f-93ea-69b998dd4a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:30.096846 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e2886-6bcb-458f-93ea-69b998dd4a47 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:30.097130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e2886-6bcb-458f-93ea-69b998dd4a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:30.097406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221e2886-6bcb-458f-93ea-69b998dd4a47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:30.107132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e2886-6bcb-458f-93ea-69b998dd4a47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:30.108635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221e2886-6bcb-458f-93ea-69b998dd4a47 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:30.110301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d015bdd9-e5f1-4d32-a446-01ed30723bed None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:30.125839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-221e2886-6bcb-458f-93ea-69b998dd4a47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:30.125839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:30.125839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:30.125839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:30.125839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:30.125839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:30.125839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:30.125839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:30.125839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:30.125839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:30.125839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:30.125839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:30.125839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:30.125839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:30.125839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:30.126785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:30.126785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:30.126785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:30.126785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:30.126785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:30.126785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:30.126785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:30.126785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:30.126785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:30.126785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:30.126785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:30.126785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:30.126785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:30.126785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:30.126785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:30.126785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:30.126785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:30.127960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:30.127960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:30.127960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:30.127960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:30.127960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 292/574] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:30.127960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e2886-6bcb-458f-93ea-69b998dd4a47 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:30.145820 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702ef2d-327f-4ccc-96b1-67ddc98d7194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:30.146630 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702ef2d-327f-4ccc-96b1-67ddc98d7194 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:30.146921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702ef2d-327f-4ccc-96b1-67ddc98d7194 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:30.147203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3702ef2d-327f-4ccc-96b1-67ddc98d7194 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:30.157681 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702ef2d-327f-4ccc-96b1-67ddc98d7194 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:30.159151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3702ef2d-327f-4ccc-96b1-67ddc98d7194 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:30.177117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-3702ef2d-327f-4ccc-96b1-67ddc98d7194 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:30.177117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:30.177117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:30.177117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:30.177117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:30.177117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:30.177117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:30.177117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:30.177117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:30.177117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:30.177117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:30.177117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:30.177117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:30.177117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:30.177117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:30.178104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:30.178104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:30.178104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:30.178104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:30.178104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:30.178104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:30.178104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:30.178104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:30.178104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:30.178104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:30.178104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:30.178104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:30.178104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:30.178104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:30.178104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:30.178104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:30.178104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:30.178985 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:30.178985 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:30.178985 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:30.178985 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:30.178985 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 283/575] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:30.179274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702ef2d-327f-4ccc-96b1-67ddc98d7194 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:30.227673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:30.239467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221e2886-6bcb-458f-93ea-69b998dd4a47 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:30.275511 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee51ca-2fac-471b-a7b3-5d2f6ae525b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:30.276216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee51ca-2fac-471b-a7b3-5d2f6ae525b1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:30.276511 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee51ca-2fac-471b-a7b3-5d2f6ae525b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:30.276794 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fee51ca-2fac-471b-a7b3-5d2f6ae525b1 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:30.277592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fee51ca-2fac-471b-a7b3-5d2f6ae525b1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:30.277915 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fee51ca-2fac-471b-a7b3-5d2f6ae525b1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:30.279432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fee51ca-2fac-471b-a7b3-5d2f6ae525b1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:30.281013 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3702ef2d-327f-4ccc-96b1-67ddc98d7194 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:30.290796 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 293/576] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:30.295936 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee51ca-2fac-471b-a7b3-5d2f6ae525b1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:30.296420 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:30.320193 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a4baf-f196-4781-86d3-a0cd1c7d396d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:30.321009 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a4baf-f196-4781-86d3-a0cd1c7d396d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:30.321296 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a4baf-f196-4781-86d3-a0cd1c7d396d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:30.321583 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378a4baf-f196-4781-86d3-a0cd1c7d396d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:30.331901 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378a4baf-f196-4781-86d3-a0cd1c7d396d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:30.340120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-378a4baf-f196-4781-86d3-a0cd1c7d396d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:30.340120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:30.340120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:30.340120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:30.340120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:30.340120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:30.340120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:30.340120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:30.340120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:30.340120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:30.340120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:30.340120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:30.340120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:30.340120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:30.340120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:30.340120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:30.341331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:30.341331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:30.341331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:30.341331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:30.341331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:30.341331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:30.341331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:30.341331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:30.341331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:30.341331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:30.341331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:30.341331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:30.341331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:30.341331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:30.341331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:30.341331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:30.342387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:30.342387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:30.342387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:30.342387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:30.342387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:30.342387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:30.342387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:30.342387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:30.342387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:30.342387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:30.342387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:30.342387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:30.342387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 284/577] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:30 2022] GET /identity/v3/roles/service => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:30.342387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a4baf-f196-4781-86d3-a0cd1c7d396d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:30.398227 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fee51ca-2fac-471b-a7b3-5d2f6ae525b1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:30.434754 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d07fc-e7a1-423c-8428-a83081685f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:30.436190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d07fc-e7a1-423c-8428-a83081685f83 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:30.436534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d07fc-e7a1-423c-8428-a83081685f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:30.436841 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518d07fc-e7a1-423c-8428-a83081685f83 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:30.443853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d07fc-e7a1-423c-8428-a83081685f83 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:30.444451 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378a4baf-f196-4781-86d3-a0cd1c7d396d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:30.445366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518d07fc-e7a1-423c-8428-a83081685f83 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:30.463432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-518d07fc-e7a1-423c-8428-a83081685f83 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:30.463432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:30.463432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:30.463432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:30.463432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:30.463432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:30.463432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:30.463432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:30.463432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:30.463432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:30.463432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:30.463432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:30.463432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:30.463432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:30.463432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:30.464424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:30.464424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:30.464424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:30.464424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:30.464424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:30.464424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:30.464424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:30.464424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:30.464424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:30.464424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:30.464424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:30.464424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:30.464424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:30.464424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:30.464424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:30.464424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:30.464424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:30.465316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:30.465316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:30.465316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:30.465316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:30.465316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:30.465316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:30.465316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:30.465316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:30.465316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 294/578] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:30.466312 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d07fc-e7a1-423c-8428-a83081685f83 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:30.482236 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:30.485800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22bc23-f317-413e-b089-a5aa0da8bcb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:30.486497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22bc23-f317-413e-b089-a5aa0da8bcb3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:30.486780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22bc23-f317-413e-b089-a5aa0da8bcb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:30.487054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d22bc23-f317-413e-b089-a5aa0da8bcb3 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:30.487776 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22bc23-f317-413e-b089-a5aa0da8bcb3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:30.488123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22bc23-f317-413e-b089-a5aa0da8bcb3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:30.489529 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d22bc23-f317-413e-b089-a5aa0da8bcb3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:30.501517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 285/579] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:30.502304 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22bc23-f317-413e-b089-a5aa0da8bcb3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:30.533120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:30.566346 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518d07fc-e7a1-423c-8428-a83081685f83 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:30.600713 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d22bc23-f317-413e-b089-a5aa0da8bcb3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:30.602016 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b8e9e-4f17-4cf9-bdc2-ab5722c332aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:30.602694 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b8e9e-4f17-4cf9-bdc2-ab5722c332aa None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:30.602987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b8e9e-4f17-4cf9-bdc2-ab5722c332aa None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:30.603271 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034b8e9e-4f17-4cf9-bdc2-ab5722c332aa None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:30.604040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b8e9e-4f17-4cf9-bdc2-ab5722c332aa None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:30.604358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b8e9e-4f17-4cf9-bdc2-ab5722c332aa None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:30.605764 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034b8e9e-4f17-4cf9-bdc2-ab5722c332aa None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:30.617312 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 295/580] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:30.618212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b8e9e-4f17-4cf9-bdc2-ab5722c332aa None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:30.636073 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ff9e6-3939-4792-9bf9-ddcf97ba35fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:30.636736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ff9e6-3939-4792-9bf9-ddcf97ba35fa None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:30.637025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ff9e6-3939-4792-9bf9-ddcf97ba35fa None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:30.637306 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1ff9e6-3939-4792-9bf9-ddcf97ba35fa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:30.638222 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ff9e6-3939-4792-9bf9-ddcf97ba35fa None admin] RBAC: Adding query filter params (role.id=83bf7d9c29d4464dab8030f6d706cd6d, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=77fda27641c84fccb68a5cdaf4e8dc4b) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:30.638529 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ff9e6-3939-4792-9bf9-ddcf97ba35fa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:30.640020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1ff9e6-3939-4792-9bf9-ddcf97ba35fa None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:30.645484 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 286/581] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:54:30 2022] GET /identity/v3/role_assignments?user.id=77fda27641c84fccb68a5cdaf4e8dc4b&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=83bf7d9c29d4464dab8030f6d706cd6d => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:30.646267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ff9e6-3939-4792-9bf9-ddcf97ba35fa None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:30.703475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:30.734340 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034b8e9e-4f17-4cf9-bdc2-ab5722c332aa None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:30.767731 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:30.774023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788c715-ba10-4ae3-a396-d965ea7e0267 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:30.774725 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788c715-ba10-4ae3-a396-d965ea7e0267 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:30.775017 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788c715-ba10-4ae3-a396-d965ea7e0267 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:30.775119 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1ff9e6-3939-4792-9bf9-ddcf97ba35fa None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:30.775307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a788c715-ba10-4ae3-a396-d965ea7e0267 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:30.776100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788c715-ba10-4ae3-a396-d965ea7e0267 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:30.776447 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788c715-ba10-4ae3-a396-d965ea7e0267 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:30.778069 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a788c715-ba10-4ae3-a396-d965ea7e0267 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:30.787598 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 296/582] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:30 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:30.787598 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788c715-ba10-4ae3-a396-d965ea7e0267 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:30.816555 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329298e-f5ce-40de-a3b7-35eab4f660c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:30.817354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329298e-f5ce-40de-a3b7-35eab4f660c5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:30.817645 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329298e-f5ce-40de-a3b7-35eab4f660c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:30.817933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329298e-f5ce-40de-a3b7-35eab4f660c5 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:30.818795 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329298e-f5ce-40de-a3b7-35eab4f660c5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:30.819118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329298e-f5ce-40de-a3b7-35eab4f660c5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:30.820633 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7329298e-f5ce-40de-a3b7-35eab4f660c5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:30.834764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 287/583] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:30.835532 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329298e-f5ce-40de-a3b7-35eab4f660c5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:30.914877 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a788c715-ba10-4ae3-a396-d965ea7e0267 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:30.950613 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899b8d4-db28-4c62-bb99-8ef53c8c31c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:30.951429 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899b8d4-db28-4c62-bb99-8ef53c8c31c8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:30.951721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899b8d4-db28-4c62-bb99-8ef53c8c31c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:30.952033 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0899b8d4-db28-4c62-bb99-8ef53c8c31c8 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:30.953015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329298e-f5ce-40de-a3b7-35eab4f660c5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:30.956105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:30.962668 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0899b8d4-db28-4c62-bb99-8ef53c8c31c8 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:30.964271 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0899b8d4-db28-4c62-bb99-8ef53c8c31c8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:30.979971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0899b8d4-db28-4c62-bb99-8ef53c8c31c8 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 15:54:30.979971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:30.979971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:30.979971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:30.979971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:30.979971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:30.979971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:30.979971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:30.979971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:30.979971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:30.979971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:30.979971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:30.979971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:30.979971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:30.979971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:30.979971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:30.980979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:30.981989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:30.981989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:30.981989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:30.981989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:30.981989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:30.981989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:30.981989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:30.981989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:30.981989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 15:54:30.981989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:30.981989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 297/584] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:30 2022] GET /identity/v3/users/cinder => generated 84 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:30.981989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899b8d4-db28-4c62-bb99-8ef53c8c31c8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:30.993999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94d067-9cb1-4998-adc8-da45be8946f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:30.994837 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d067-9cb1-4998-adc8-da45be8946f2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:30.995123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d067-9cb1-4998-adc8-da45be8946f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:30.995381 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a94d067-9cb1-4998-adc8-da45be8946f2 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:31.003158 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94d067-9cb1-4998-adc8-da45be8946f2 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:31.004680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a94d067-9cb1-4998-adc8-da45be8946f2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:31.019635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-4a94d067-9cb1-4998-adc8-da45be8946f2 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 15:54:31.019635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:31.019635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:31.019635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:31.019635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:31.019635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:31.019635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:31.019635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:31.019635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:31.019635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:31.019635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:31.019635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:31.019635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:31.019635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:31.019635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:31.019635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:31.020732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:31.021959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:31.021959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:31.021959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:31.021959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:31.021959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:31.021959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:31.021959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:31.021959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:31.021959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 15:54:31.021959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:31.025359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 288/585] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:30 2022] GET /identity/v3/users/glance => generated 84 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:31.026245 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94d067-9cb1-4998-adc8-da45be8946f2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:31.044128 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:31.094091 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0899b8d4-db28-4c62-bb99-8ef53c8c31c8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:31.130300 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2d45c-382e-44a5-8b50-10938de6313e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:31.131412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2d45c-382e-44a5-8b50-10938de6313e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:31.131920 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2d45c-382e-44a5-8b50-10938de6313e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:31.132207 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec2d45c-382e-44a5-8b50-10938de6313e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:31.133452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a94d067-9cb1-4998-adc8-da45be8946f2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:31.143431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2d45c-382e-44a5-8b50-10938de6313e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:31.144944 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec2d45c-382e-44a5-8b50-10938de6313e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:31.161829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-cec2d45c-382e-44a5-8b50-10938de6313e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:31.161829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:31.161829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:31.161829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:31.161829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:31.161829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:31.161829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:31.161829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:31.161829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:31.161829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:31.161829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:31.161829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:31.161829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:31.161829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:31.161829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:31.162982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:31.162982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:31.162982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:31.162982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:31.162982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:31.162982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:31.162982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:31.162982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:31.162982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:31.162982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:31.162982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:31.162982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:31.162982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:31.162982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:31.162982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:31.162982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:31.162982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:31.168738 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:31.168738 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:31.168738 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:31.168738 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:31.168738 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 298/586] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:31.168738 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2d45c-382e-44a5-8b50-10938de6313e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:31.177420 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb557486-c169-41a9-a406-9e8d6658d202 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:31.178489 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb557486-c169-41a9-a406-9e8d6658d202 None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:31.178781 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb557486-c169-41a9-a406-9e8d6658d202 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:31.179059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb557486-c169-41a9-a406-9e8d6658d202 None admin] PATH_INFO: `/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/4751abc9f67947b3ad17ba2b78ce2c12/roles/83bf7d9c29d4464dab8030f6d706cd6d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:31.196879 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb557486-c169-41a9-a406-9e8d6658d202 None admin] RBAC: Authorizing `identity:create_grant(project_id=5584ebdaf133444ca69efc5554bf48e9, user_id=4751abc9f67947b3ad17ba2b78ce2c12, role_id=83bf7d9c29d4464dab8030f6d706cd6d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:31.198360 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb557486-c169-41a9-a406-9e8d6658d202 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:31.212103 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 289/587] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:54:31 2022] PUT /identity/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/4751abc9f67947b3ad17ba2b78ce2c12/roles/83bf7d9c29d4464dab8030f6d706cd6d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:54:31.213329 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb557486-c169-41a9-a406-9e8d6658d202 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:31.233843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:31.305945 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec2d45c-382e-44a5-8b50-10938de6313e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:31.306502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:31.350917 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0d6d2-e217-45b8-bb02-12323cf5c394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:31.351996 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0d6d2-e217-45b8-bb02-12323cf5c394 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:31.352481 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0d6d2-e217-45b8-bb02-12323cf5c394 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:31.353041 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca0d6d2-e217-45b8-bb02-12323cf5c394 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:31.354191 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca0d6d2-e217-45b8-bb02-12323cf5c394 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:31.354676 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca0d6d2-e217-45b8-bb02-12323cf5c394 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:31.356391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca0d6d2-e217-45b8-bb02-12323cf5c394 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:31.356524 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb557486-c169-41a9-a406-9e8d6658d202 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:31.382769 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 299/588] 10.222.0.85 () {56 vars in 1137 bytes} [Wed May 25 15:54:31 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:31.384104 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0d6d2-e217-45b8-bb02-12323cf5c394 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:31.402713 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56509ad-218f-47b9-a296-82629bfce39f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:31.403582 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56509ad-218f-47b9-a296-82629bfce39f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:31.403912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56509ad-218f-47b9-a296-82629bfce39f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:31.404201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56509ad-218f-47b9-a296-82629bfce39f None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:31.405363 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56509ad-218f-47b9-a296-82629bfce39f None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:31.405689 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56509ad-218f-47b9-a296-82629bfce39f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:31.407670 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56509ad-218f-47b9-a296-82629bfce39f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:31.435254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 290/589] 10.222.0.85 () {56 vars in 1137 bytes} [Wed May 25 15:54:31 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:31.435254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56509ad-218f-47b9-a296-82629bfce39f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:31.520824 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca0d6d2-e217-45b8-bb02-12323cf5c394 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:31.529106 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:31.557850 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a182ae-8b51-44dc-b039-4f56968d2106 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:31.558532 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a182ae-8b51-44dc-b039-4f56968d2106 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:31.558821 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a182ae-8b51-44dc-b039-4f56968d2106 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:31.559104 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a182ae-8b51-44dc-b039-4f56968d2106 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:31.559929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a182ae-8b51-44dc-b039-4f56968d2106 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:31.560406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a182ae-8b51-44dc-b039-4f56968d2106 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:31.560928 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56509ad-218f-47b9-a296-82629bfce39f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:31.561938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a182ae-8b51-44dc-b039-4f56968d2106 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:31.577682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 300/590] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:31.578559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a182ae-8b51-44dc-b039-4f56968d2106 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:31.590387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:31.603440 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ca901-2fa1-4af7-ad05-38eb1c1e1f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:31.604328 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ca901-2fa1-4af7-ad05-38eb1c1e1f8f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:31.604624 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ca901-2fa1-4af7-ad05-38eb1c1e1f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:31.604921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf3ca901-2fa1-4af7-ad05-38eb1c1e1f8f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:31.615753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ca901-2fa1-4af7-ad05-38eb1c1e1f8f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:31.617312 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf3ca901-2fa1-4af7-ad05-38eb1c1e1f8f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:31.635682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-bf3ca901-2fa1-4af7-ad05-38eb1c1e1f8f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:31.635682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:31.635682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:31.635682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:31.635682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:31.635682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:31.635682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:31.635682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:31.635682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:31.635682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:31.635682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:31.635682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:31.635682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:31.635682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:31.635682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:31.636667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:31.636667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:31.636667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:31.636667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:31.636667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:31.636667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:31.636667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:31.636667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:31.636667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:31.636667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:31.636667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:31.636667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:31.636667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:31.636667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:31.636667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:31.636667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:31.636667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:31.637560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:31.637560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:31.637560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:31.637560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:31.637560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 291/591] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:31.638445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ca901-2fa1-4af7-ad05-38eb1c1e1f8f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:31.694605 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a182ae-8b51-44dc-b039-4f56968d2106 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:31.729369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e44f94-680d-43d0-8875-aeb525d6227c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:31.730186 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e44f94-680d-43d0-8875-aeb525d6227c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:31.730473 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e44f94-680d-43d0-8875-aeb525d6227c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:31.730753 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41e44f94-680d-43d0-8875-aeb525d6227c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:31.742686 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e44f94-680d-43d0-8875-aeb525d6227c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:31.743015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf3ca901-2fa1-4af7-ad05-38eb1c1e1f8f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:31.744250 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41e44f94-680d-43d0-8875-aeb525d6227c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:31.763124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-41e44f94-680d-43d0-8875-aeb525d6227c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:31.763124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:31.763124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:31.763124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:31.763124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:31.763124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:31.763124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:31.763124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:31.763124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:31.763124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:31.763124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:31.763124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:31.763124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:31.763124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:31.763124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:31.765237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:31.765237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:31.765237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:31.765237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:31.765237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:31.765237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:31.765237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:31.765237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:31.765237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:31.765237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:31.765237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:31.765237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:31.765237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:31.765237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:31.765237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:31.765237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:31.765237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:31.767358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:31.767358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:31.767358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:31.767358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:31.767358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 301/592] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:31.767358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e44f94-680d-43d0-8875-aeb525d6227c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:31.780772 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9597d36-3d1c-4ec5-96cd-315706b0d762 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:31.781512 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9597d36-3d1c-4ec5-96cd-315706b0d762 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:31.781804 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9597d36-3d1c-4ec5-96cd-315706b0d762 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:31.782086 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9597d36-3d1c-4ec5-96cd-315706b0d762 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:31.786276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:31.791245 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9597d36-3d1c-4ec5-96cd-315706b0d762 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:31.792943 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9597d36-3d1c-4ec5-96cd-315706b0d762 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:31.811980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-f9597d36-3d1c-4ec5-96cd-315706b0d762 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 15:54:31.811980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:31.811980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:31.811980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:31.811980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:31.811980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:31.811980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:31.811980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:31.811980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:31.811980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:31.811980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:31.811980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:31.811980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:31.811980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:31.811980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:31.811980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:31.813095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:31.814026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:31.814026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:31.814026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:31.814026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:31.814026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:31.814026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:31.814026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:31.814026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:31.814026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 15:54:31.814026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:31.814026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 292/593] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:31 2022] GET /identity/v3/users/neutron => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:31.814624 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9597d36-3d1c-4ec5-96cd-315706b0d762 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:31.855602 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:31.884841 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41e44f94-680d-43d0-8875-aeb525d6227c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:31.921136 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fdf68-4ac6-4c18-9889-c0ff69e0fa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:31.922096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fdf68-4ac6-4c18-9889-c0ff69e0fa6a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:31.922343 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fdf68-4ac6-4c18-9889-c0ff69e0fa6a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:31.922689 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448fdf68-4ac6-4c18-9889-c0ff69e0fa6a None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:31.923513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fdf68-4ac6-4c18-9889-c0ff69e0fa6a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:31.923893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fdf68-4ac6-4c18-9889-c0ff69e0fa6a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:31.925509 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448fdf68-4ac6-4c18-9889-c0ff69e0fa6a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:31.931858 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9597d36-3d1c-4ec5-96cd-315706b0d762 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:31.938604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 302/594] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:31.939992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fdf68-4ac6-4c18-9889-c0ff69e0fa6a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:31.968448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ceb1a-e5f6-4c1a-844f-0b89a8e07d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:31.969538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ceb1a-e5f6-4c1a-844f-0b89a8e07d01 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:31.970015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ceb1a-e5f6-4c1a-844f-0b89a8e07d01 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:31.970473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6ceb1a-e5f6-4c1a-844f-0b89a8e07d01 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:31.971479 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ceb1a-e5f6-4c1a-844f-0b89a8e07d01 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:31.972013 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ceb1a-e5f6-4c1a-844f-0b89a8e07d01 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:31.973670 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6ceb1a-e5f6-4c1a-844f-0b89a8e07d01 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:31.987533 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 293/595] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:31.988745 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ceb1a-e5f6-4c1a-844f-0b89a8e07d01 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:32.023379 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:32.047799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448fdf68-4ac6-4c18-9889-c0ff69e0fa6a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:32.086013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00209115-defb-4df3-aa67-65ce9e7d07ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:32.086791 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00209115-defb-4df3-aa67-65ce9e7d07ab None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:32.087240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00209115-defb-4df3-aa67-65ce9e7d07ab None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:32.087581 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00209115-defb-4df3-aa67-65ce9e7d07ab None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:32.088642 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00209115-defb-4df3-aa67-65ce9e7d07ab None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:32.089027 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00209115-defb-4df3-aa67-65ce9e7d07ab None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:32.090637 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00209115-defb-4df3-aa67-65ce9e7d07ab None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:32.091294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:32.098281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6ceb1a-e5f6-4c1a-844f-0b89a8e07d01 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:32.111441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 303/596] 10.222.0.85 () {56 vars in 1139 bytes} [Wed May 25 15:54:31 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:32.112337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00209115-defb-4df3-aa67-65ce9e7d07ab None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:32.135513 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fa8ab-c5a0-43eb-8214-95d50072cd6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:32.136852 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fa8ab-c5a0-43eb-8214-95d50072cd6c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:32.137377 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fa8ab-c5a0-43eb-8214-95d50072cd6c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:32.137869 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26fa8ab-c5a0-43eb-8214-95d50072cd6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:32.145713 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26fa8ab-c5a0-43eb-8214-95d50072cd6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:32.147457 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26fa8ab-c5a0-43eb-8214-95d50072cd6c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:32.162379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-e26fa8ab-c5a0-43eb-8214-95d50072cd6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:32.162379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:32.162379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:32.162379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:32.162379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:32.162379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:32.162379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:32.162379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:32.162379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:32.162379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:32.162379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:32.162379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:32.162379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:32.162379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:32.162379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:32.163281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:32.163281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:32.163281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:32.163281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:32.163281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:32.163281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:32.163281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:32.163281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:32.163281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:32.163281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:32.163281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:32.163281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:32.163281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:32.163281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:32.163281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:32.163281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:32.163281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:32.164202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:32.164202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:32.164202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:32.164202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:32.164202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:32.164202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:32.164202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:32.164202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:32.166705 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 294/597] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:32.167783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fa8ab-c5a0-43eb-8214-95d50072cd6c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:32.221239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00209115-defb-4df3-aa67-65ce9e7d07ab None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:32.258369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbcf03-439f-41e9-bc13-e208622adb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:32.259181 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbcf03-439f-41e9-bc13-e208622adb0b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:32.259470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbcf03-439f-41e9-bc13-e208622adb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:32.259802 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fbcf03-439f-41e9-bc13-e208622adb0b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:32.262833 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:32.268922 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbcf03-439f-41e9-bc13-e208622adb0b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:32.270559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fbcf03-439f-41e9-bc13-e208622adb0b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:32.275516 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26fa8ab-c5a0-43eb-8214-95d50072cd6c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:32.285404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-79fbcf03-439f-41e9-bc13-e208622adb0b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:32.285404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:32.285404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:32.285404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:32.285404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:32.285404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:32.285404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:32.285404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:32.285404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:32.285404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:32.285404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:32.285404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:32.285404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:32.285404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:32.285404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:32.286493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:32.286493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:32.286493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:32.286493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:32.286493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:32.286493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:32.286493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:32.286493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:32.286493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:32.286493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:32.286493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:32.286493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:32.286493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:32.286493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:32.286493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:32.286493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:32.286493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:32.287670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:32.287670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:32.287670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:32.287670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:32.287670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:32.287670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:32.287670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:32.287670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:32.287670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 304/598] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:32.288212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbcf03-439f-41e9-bc13-e208622adb0b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:32.328625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df408a4c-3406-4ef0-a440-3373a4639f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:32.329779 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df408a4c-3406-4ef0-a440-3373a4639f0f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:32.330211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df408a4c-3406-4ef0-a440-3373a4639f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:32.330657 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df408a4c-3406-4ef0-a440-3373a4639f0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:32.344324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df408a4c-3406-4ef0-a440-3373a4639f0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:32.346158 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df408a4c-3406-4ef0-a440-3373a4639f0f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:32.364853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-df408a4c-3406-4ef0-a440-3373a4639f0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:32.364853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:32.364853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:32.364853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:32.364853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:32.364853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:32.364853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:32.364853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:32.364853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:32.364853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:32.364853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:32.364853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:32.364853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:32.364853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:32.364853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:32.365989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:32.365989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:32.365989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:32.365989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:32.365989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:32.365989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:32.365989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:32.365989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:32.365989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:32.365989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:32.365989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:32.365989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:32.365989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:32.365989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:32.365989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:32.365989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:32.365989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:32.367085 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:32.367085 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:32.367085 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:32.367085 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:32.368926 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 295/599] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:32.370014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df408a4c-3406-4ef0-a440-3373a4639f0f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:32.380402 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:32.397039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fbcf03-439f-41e9-bc13-e208622adb0b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:32.433496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaeadd9-3856-4270-b6ed-738c5870ce8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:32.434663 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeadd9-3856-4270-b6ed-738c5870ce8e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:32.434777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeadd9-3856-4270-b6ed-738c5870ce8e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:32.435058 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abaeadd9-3856-4270-b6ed-738c5870ce8e None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:32.435951 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaeadd9-3856-4270-b6ed-738c5870ce8e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:32.436309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaeadd9-3856-4270-b6ed-738c5870ce8e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:32.437929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abaeadd9-3856-4270-b6ed-738c5870ce8e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:32.452917 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 305/600] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:32.453869 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaeadd9-3856-4270-b6ed-738c5870ce8e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:32.493611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df408a4c-3406-4ef0-a440-3373a4639f0f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:32.515257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:32.535497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b8c4f-9710-419a-8aa9-8e6a613f250b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:32.536669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b8c4f-9710-419a-8aa9-8e6a613f250b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:32.537158 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b8c4f-9710-419a-8aa9-8e6a613f250b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:32.537663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a47b8c4f-9710-419a-8aa9-8e6a613f250b None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:32.538716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b8c4f-9710-419a-8aa9-8e6a613f250b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:32.539230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b8c4f-9710-419a-8aa9-8e6a613f250b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:32.541198 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a47b8c4f-9710-419a-8aa9-8e6a613f250b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:32.556627 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 296/601] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:32 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:32.558554 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b8c4f-9710-419a-8aa9-8e6a613f250b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:32.572303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abaeadd9-3856-4270-b6ed-738c5870ce8e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:32.614507 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a68dc6-31a5-47ad-8b84-0855110a2f58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:32.615192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a68dc6-31a5-47ad-8b84-0855110a2f58 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:32.615570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a68dc6-31a5-47ad-8b84-0855110a2f58 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:32.615971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37a68dc6-31a5-47ad-8b84-0855110a2f58 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:32.616833 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a68dc6-31a5-47ad-8b84-0855110a2f58 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:32.617152 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a68dc6-31a5-47ad-8b84-0855110a2f58 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:32.618558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37a68dc6-31a5-47ad-8b84-0855110a2f58 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:32.634199 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 306/602] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:32.635671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a68dc6-31a5-47ad-8b84-0855110a2f58 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:32.636556 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:32.678158 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a47b8c4f-9710-419a-8aa9-8e6a613f250b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:32.727632 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a01167-080b-4385-b13a-5a4f0d23b11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:32.729900 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a01167-080b-4385-b13a-5a4f0d23b11a None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:32.730749 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a01167-080b-4385-b13a-5a4f0d23b11a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:32.731548 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6a01167-080b-4385-b13a-5a4f0d23b11a None admin] PATH_INFO: `/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/f81aecdc87d84501ba94b208405b51dd/roles/83bf7d9c29d4464dab8030f6d706cd6d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:32.754795 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a01167-080b-4385-b13a-5a4f0d23b11a None admin] RBAC: Authorizing `identity:create_grant(project_id=5584ebdaf133444ca69efc5554bf48e9, user_id=f81aecdc87d84501ba94b208405b51dd, role_id=83bf7d9c29d4464dab8030f6d706cd6d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:32.756840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6a01167-080b-4385-b13a-5a4f0d23b11a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:32.763761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37a68dc6-31a5-47ad-8b84-0855110a2f58 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:32.771989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 297/603] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:54:32 2022] PUT /identity/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/f81aecdc87d84501ba94b208405b51dd/roles/83bf7d9c29d4464dab8030f6d706cd6d => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:54:32.773286 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a01167-080b-4385-b13a-5a4f0d23b11a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:32.784307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:32.807350 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651cd05d-f938-4572-bb95-e52185b40625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:32.808217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cd05d-f938-4572-bb95-e52185b40625 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:32.808507 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cd05d-f938-4572-bb95-e52185b40625 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:32.808773 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651cd05d-f938-4572-bb95-e52185b40625 None admin] PATH_INFO: `/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/196dc406a95444c39babc50d59f9eb6b/roles/83bf7d9c29d4464dab8030f6d706cd6d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:32.831092 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651cd05d-f938-4572-bb95-e52185b40625 None admin] RBAC: Authorizing `identity:create_grant(project_id=5584ebdaf133444ca69efc5554bf48e9, user_id=196dc406a95444c39babc50d59f9eb6b, role_id=83bf7d9c29d4464dab8030f6d706cd6d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:32.833079 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651cd05d-f938-4572-bb95-e52185b40625 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:32.872269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 307/604] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:54:32 2022] PUT /identity/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/196dc406a95444c39babc50d59f9eb6b/roles/83bf7d9c29d4464dab8030f6d706cd6d => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:54:32.924116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380a347-8824-4f96-ba54-8070f7ecb297 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:32.924611 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380a347-8824-4f96-ba54-8070f7ecb297 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:32.925171 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7380a347-8824-4f96-ba54-8070f7ecb297 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:32.927677 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6a01167-080b-4385-b13a-5a4f0d23b11a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:32.934552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 308/605] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:54:32.937385 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b339e3e-eb78-4409-a2d7-775e9b4f3856 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:32.937877 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b339e3e-eb78-4409-a2d7-775e9b4f3856 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:32.938357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b339e3e-eb78-4409-a2d7-775e9b4f3856 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:32.971976 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:33.007034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c106cbe-ac86-4da1-a069-7866033e1c44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:33.009033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c106cbe-ac86-4da1-a069-7866033e1c44 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:33.009570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4b339e3e-eb78-4409-a2d7-775e9b4f3856 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:33.010096 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c106cbe-ac86-4da1-a069-7866033e1c44 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:33.010834 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c106cbe-ac86-4da1-a069-7866033e1c44 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:33.011566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4b339e3e-eb78-4409-a2d7-775e9b4f3856 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:33.024614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c106cbe-ac86-4da1-a069-7866033e1c44 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:33.031708 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c106cbe-ac86-4da1-a069-7866033e1c44 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:33.071796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-5c106cbe-ac86-4da1-a069-7866033e1c44 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:33.071796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:33.071796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:33.071796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:33.071796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:33.071796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:33.071796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:33.071796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:33.071796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:33.071796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:33.071796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:33.071796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:33.071796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:33.071796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:33.071796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:33.073391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:33.073391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:33.073391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:33.073391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:33.073391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:33.073391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:33.073391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:33.073391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:33.073391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:33.073391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:33.073391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:33.073391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:33.073391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:33.073391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:33.073391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:33.073391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:33.073391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:33.074966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:33.074966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:33.074966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:33.074966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:33.074966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:33.074966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:33.074966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:33.074966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:33.077543 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 298/606] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 305 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:33.093035 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c106cbe-ac86-4da1-a069-7866033e1c44 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:33.148038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 309/607] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:54:33.202943 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b339e3e-eb78-4409-a2d7-775e9b4f3856 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:33.344003 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:33.345918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c106cbe-ac86-4da1-a069-7866033e1c44 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:33.353786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b339e3e-eb78-4409-a2d7-775e9b4f3856 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:33.398157 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a80da9-660b-40f7-be48-83b1b1751877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:33.399372 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a80da9-660b-40f7-be48-83b1b1751877 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:33.401663 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a80da9-660b-40f7-be48-83b1b1751877 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:33.402406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a80da9-660b-40f7-be48-83b1b1751877 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:33.424215 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a80da9-660b-40f7-be48-83b1b1751877 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:33.432293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f7a80da9-660b-40f7-be48-83b1b1751877 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:33.432293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:33.432293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:33.432293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:33.432293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:33.432293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:33.432293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:33.432293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:33.432293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:33.432293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:33.432293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:33.432293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:33.432293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:33.432293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:33.432293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:33.432293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:33.433228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:33.433228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:33.433228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:33.433228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:33.433228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:33.433228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:33.433228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:33.433228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:33.433228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:33.433228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:33.433228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:33.433228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:33.433228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:33.433228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:33.433228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:33.433228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:33.434138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:33.434138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:33.434138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:33.434138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:33.434138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:33.434138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:33.434138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:33.434138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:33.434138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:33.434138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:33.434138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:33.434138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:33.434914 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2185eb-610a-4bad-8b93-28db5bb07900 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:33.434914 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2185eb-610a-4bad-8b93-28db5bb07900 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:33.434914 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2185eb-610a-4bad-8b93-28db5bb07900 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:33.434914 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2185eb-610a-4bad-8b93-28db5bb07900 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:33.435200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 310/608] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:33.435276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2185eb-610a-4bad-8b93-28db5bb07900 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:33.435276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2185eb-610a-4bad-8b93-28db5bb07900 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:33.439033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2185eb-610a-4bad-8b93-28db5bb07900 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:33.440102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a80da9-660b-40f7-be48-83b1b1751877 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:33.460731 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 299/609] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 368 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:33.468833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2185eb-610a-4bad-8b93-28db5bb07900 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:33.580121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:33.603802 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a80da9-660b-40f7-be48-83b1b1751877 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:33.620975 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2185eb-610a-4bad-8b93-28db5bb07900 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:33.675327 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625de81b-ac24-4e10-83c9-5122789d78fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:33.676834 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625de81b-ac24-4e10-83c9-5122789d78fc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:33.677314 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625de81b-ac24-4e10-83c9-5122789d78fc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:33.677806 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625de81b-ac24-4e10-83c9-5122789d78fc None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:33.678752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625de81b-ac24-4e10-83c9-5122789d78fc None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:33.679287 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625de81b-ac24-4e10-83c9-5122789d78fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:33.681007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625de81b-ac24-4e10-83c9-5122789d78fc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:33.683146 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb343c16-9d1c-4459-b345-cdd7861afeea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:33.684204 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb343c16-9d1c-4459-b345-cdd7861afeea None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:33.684497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb343c16-9d1c-4459-b345-cdd7861afeea None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:33.684808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb343c16-9d1c-4459-b345-cdd7861afeea None admin] PATH_INFO: `/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/413b3d65427b41dc99bc7b0a94fb1dc8/roles/83bf7d9c29d4464dab8030f6d706cd6d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:33.691888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 311/610] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:33.708078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625de81b-ac24-4e10-83c9-5122789d78fc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:33.708343 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb343c16-9d1c-4459-b345-cdd7861afeea None admin] RBAC: Authorizing `identity:create_grant(project_id=5584ebdaf133444ca69efc5554bf48e9, user_id=413b3d65427b41dc99bc7b0a94fb1dc8, role_id=83bf7d9c29d4464dab8030f6d706cd6d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:33.709862 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb343c16-9d1c-4459-b345-cdd7861afeea None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:33.731546 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 300/611] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:54:33 2022] PUT /identity/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/413b3d65427b41dc99bc7b0a94fb1dc8/roles/83bf7d9c29d4464dab8030f6d706cd6d => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:54:33.739219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:33.746020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bee0f5-a2a1-4fbf-b45a-d2c19c6bdaa0 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:33.747456 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bee0f5-a2a1-4fbf-b45a-d2c19c6bdaa0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:33.750591 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8bee0f5-a2a1-4fbf-b45a-d2c19c6bdaa0 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:33.753077 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 301/612] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:33 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:54:33.758807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35102e8f-0e39-4288-8e80-c443911aadba None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:33.759366 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35102e8f-0e39-4288-8e80-c443911aadba None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:33.759815 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35102e8f-0e39-4288-8e80-c443911aadba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:33.789063 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-35102e8f-0e39-4288-8e80-c443911aadba None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:33.790615 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-35102e8f-0e39-4288-8e80-c443911aadba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:33.851743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625de81b-ac24-4e10-83c9-5122789d78fc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:33.892316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb258881-7f90-4167-be28-1dbf94816454 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:33.893449 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb258881-7f90-4167-be28-1dbf94816454 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:33.893823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb258881-7f90-4167-be28-1dbf94816454 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:33.895363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb258881-7f90-4167-be28-1dbf94816454 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:33.909631 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb258881-7f90-4167-be28-1dbf94816454 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:33.911134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb258881-7f90-4167-be28-1dbf94816454 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:33.932375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-eb258881-7f90-4167-be28-1dbf94816454 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:33.932375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:33.932375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:33.932375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:33.932375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:33.932375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:33.932375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:33.932375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:33.932375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:33.932375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:33.932375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:33.932375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:33.932375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:33.932375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:33.932375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:33.933370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:33.933370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:33.933370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:33.933370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:33.933370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:33.933370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:33.933370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:33.933370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:33.933370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:33.933370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:33.933370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:33.933370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:33.933370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:33.933370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:33.933370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:33.933370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:33.933370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:33.934253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:33.934253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:33.934253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:33.934253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:33.934253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 312/613] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:54:33.940785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb258881-7f90-4167-be28-1dbf94816454 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:33.972430 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 302/614] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:33.998832 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35102e8f-0e39-4288-8e80-c443911aadba None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:34.018240 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:34.019986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:34.057609 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb258881-7f90-4167-be28-1dbf94816454 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:34.102324 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c95ab-1f03-4fc3-891b-942827d81e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:34.103099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c95ab-1f03-4fc3-891b-942827d81e7c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:34.103397 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c95ab-1f03-4fc3-891b-942827d81e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:34.103683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d12c95ab-1f03-4fc3-891b-942827d81e7c None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:34.104527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c95ab-1f03-4fc3-891b-942827d81e7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:34.104949 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c95ab-1f03-4fc3-891b-942827d81e7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:34.106439 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d12c95ab-1f03-4fc3-891b-942827d81e7c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:34.111385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35102e8f-0e39-4288-8e80-c443911aadba None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:34.129586 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 313/615] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:34.137395 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c95ab-1f03-4fc3-891b-942827d81e7c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:34.151675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0be4a-12bb-4cca-94e5-786090599351 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:34.152487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0be4a-12bb-4cca-94e5-786090599351 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:34.152777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0be4a-12bb-4cca-94e5-786090599351 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:34.153057 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f0be4a-12bb-4cca-94e5-786090599351 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:34.163580 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f0be4a-12bb-4cca-94e5-786090599351 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:34.171952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-a7f0be4a-12bb-4cca-94e5-786090599351 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:34.171952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:34.171952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:34.171952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:34.171952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:34.171952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:34.171952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:34.171952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:34.171952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:34.171952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:34.171952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:34.171952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:34.171952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:34.171952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:34.171952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:34.171952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:34.172870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:34.172870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:34.172870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:34.172870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:34.172870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:34.172870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:34.172870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:34.172870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:34.172870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:34.172870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:34.172870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:34.172870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:34.172870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:34.172870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:34.172870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:34.172870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:34.173712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:34.173712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:34.173712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:34.173712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:34.173712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:34.173712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:34.173712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:34.173712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:34.173712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:34.173712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:34.173712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:34.173712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:34.173712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 303/616] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:34.179273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0be4a-12bb-4cca-94e5-786090599351 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:34.260649 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:34.286243 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f0be4a-12bb-4cca-94e5-786090599351 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:34.324566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498398ca-23f8-46b8-98a2-c7bc1c2db003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:34.325891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498398ca-23f8-46b8-98a2-c7bc1c2db003 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:34.326188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498398ca-23f8-46b8-98a2-c7bc1c2db003 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:34.326611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-498398ca-23f8-46b8-98a2-c7bc1c2db003 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:34.327386 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498398ca-23f8-46b8-98a2-c7bc1c2db003 None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:34.327704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498398ca-23f8-46b8-98a2-c7bc1c2db003 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:34.329159 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-498398ca-23f8-46b8-98a2-c7bc1c2db003 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:34.338931 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 304/617] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:34.344824 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498398ca-23f8-46b8-98a2-c7bc1c2db003 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:34.374797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d12c95ab-1f03-4fc3-891b-942827d81e7c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:34.434427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:34.452667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-498398ca-23f8-46b8-98a2-c7bc1c2db003 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:34.457088 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9b17b-a763-4612-bc6e-f1a1c16e6e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:34.458270 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9b17b-a763-4612-bc6e-f1a1c16e6e08 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:34.458562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9b17b-a763-4612-bc6e-f1a1c16e6e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:34.458848 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a9b17b-a763-4612-bc6e-f1a1c16e6e08 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:34.480908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9b17b-a763-4612-bc6e-f1a1c16e6e08 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:34.482467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a9b17b-a763-4612-bc6e-f1a1c16e6e08 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:34.488666 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49e06a-6bb6-431d-9b9a-ab969b52e16a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:34.489482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49e06a-6bb6-431d-9b9a-ab969b52e16a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:34.489767 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49e06a-6bb6-431d-9b9a-ab969b52e16a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:34.490043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49e06a-6bb6-431d-9b9a-ab969b52e16a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:34.501101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49e06a-6bb6-431d-9b9a-ab969b52e16a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:34.502696 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49e06a-6bb6-431d-9b9a-ab969b52e16a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:34.506117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a3a9b17b-a763-4612-bc6e-f1a1c16e6e08 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 15:54:34.506117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:34.506117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:34.506117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:34.506117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:34.506117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:34.506117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:34.506117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:34.506117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:34.506117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:34.506117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:34.506117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:34.506117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:34.506117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:34.506117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:34.506117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:34.507046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:34.508024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:34.508024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:34.508024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:34.508024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:34.508024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:34.508024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:34.508024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:34.508024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:34.508024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 15:54:34.508024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:34.508024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 314/618] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:54:34 2022] GET /identity/v3/users/swift => generated 83 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:34.520842 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9b17b-a763-4612-bc6e-f1a1c16e6e08 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:34.525482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-0d49e06a-6bb6-431d-9b9a-ab969b52e16a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:34.525482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:34.525482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:34.525482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:34.525482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:34.525482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:34.525482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:34.525482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:34.525482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:34.525482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:34.525482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:34.525482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:34.525482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:34.525482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:34.525482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:34.526355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:34.526355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:34.526355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:34.526355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:34.526355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:34.526355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:34.526355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:34.526355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:34.526355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:34.526355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:34.526355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:34.526355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:34.526355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:34.526355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:34.526355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:34.526355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:34.526355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:34.527262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:34.527262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:34.527262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:34.527262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:34.527262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 305/619] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:34.528271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:34.532813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49e06a-6bb6-431d-9b9a-ab969b52e16a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:34.641086 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49e06a-6bb6-431d-9b9a-ab969b52e16a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:34.676576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabaf04-3451-4608-b23d-a29e72b46502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:34.677250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabaf04-3451-4608-b23d-a29e72b46502 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:34.677538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabaf04-3451-4608-b23d-a29e72b46502 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:34.677842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabaf04-3451-4608-b23d-a29e72b46502 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:34.678643 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabaf04-3451-4608-b23d-a29e72b46502 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:34.678960 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabaf04-3451-4608-b23d-a29e72b46502 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:34.680487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabaf04-3451-4608-b23d-a29e72b46502 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:34.692960 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 306/620] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:34.698248 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabaf04-3451-4608-b23d-a29e72b46502 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:34.722083 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a9b17b-a763-4612-bc6e-f1a1c16e6e08 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:34.763121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:34.785923 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99224c8-00eb-43f0-81c4-b856206197d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:34.786616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99224c8-00eb-43f0-81c4-b856206197d3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:34.786912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99224c8-00eb-43f0-81c4-b856206197d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:34.787199 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99224c8-00eb-43f0-81c4-b856206197d3 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:34.788086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99224c8-00eb-43f0-81c4-b856206197d3 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:34.788408 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99224c8-00eb-43f0-81c4-b856206197d3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:34.789896 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99224c8-00eb-43f0-81c4-b856206197d3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:34.801167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabaf04-3451-4608-b23d-a29e72b46502 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:34.825573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 315/621] 10.222.0.85 () {56 vars in 1135 bytes} [Wed May 25 15:54:34 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:34.836944 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99224c8-00eb-43f0-81c4-b856206197d3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:34.841547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e62511-6899-4f2c-8ebe-ada466765a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:34.842373 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e62511-6899-4f2c-8ebe-ada466765a86 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:34.842664 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e62511-6899-4f2c-8ebe-ada466765a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:34.842948 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e62511-6899-4f2c-8ebe-ada466765a86 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:34.851155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e62511-6899-4f2c-8ebe-ada466765a86 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:34.852807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e62511-6899-4f2c-8ebe-ada466765a86 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:34.868076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-99e62511-6899-4f2c-8ebe-ada466765a86 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 15:54:34.868076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:34.868076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:34.868076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:34.868076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:34.868076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:34.868076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:34.868076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:34.868076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:34.868076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:34.868076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:34.868076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:34.868076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:34.868076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:34.868076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:34.868076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:34.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:34.869988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:34.869988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:34.869988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:34.869988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:34.869988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:34.869988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:34.869988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:34.869988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:34.869988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 15:54:34.869988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:34.869988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 307/622] 10.222.0.85 () {56 vars in 1093 bytes} [Wed May 25 15:54:34 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:54:34.875096 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e62511-6899-4f2c-8ebe-ada466765a86 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:34.949841 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:34.991253 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e62511-6899-4f2c-8ebe-ada466765a86 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:35.014381 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:35.063196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99224c8-00eb-43f0-81c4-b856206197d3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:35.069739 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f56c1-427b-4baa-8e4c-423e88fbc307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:35.070837 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f56c1-427b-4baa-8e4c-423e88fbc307 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:35.074234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f56c1-427b-4baa-8e4c-423e88fbc307 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:35.074677 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801f56c1-427b-4baa-8e4c-423e88fbc307 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:35.075816 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f56c1-427b-4baa-8e4c-423e88fbc307 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:35.076303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f56c1-427b-4baa-8e4c-423e88fbc307 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:35.079632 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801f56c1-427b-4baa-8e4c-423e88fbc307 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:35.118522 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 308/623] 10.222.0.85 () {56 vars in 1133 bytes} [Wed May 25 15:54:34 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:35.120809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f56c1-427b-4baa-8e4c-423e88fbc307 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:35.142050 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b099de-ce94-40eb-937b-40b73feb9d63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:35.143162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b099de-ce94-40eb-937b-40b73feb9d63 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:35.144097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b099de-ce94-40eb-937b-40b73feb9d63 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:35.145643 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b099de-ce94-40eb-937b-40b73feb9d63 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:35.169957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b099de-ce94-40eb-937b-40b73feb9d63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:35.171763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b099de-ce94-40eb-937b-40b73feb9d63 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:35.211914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b1b099de-ce94-40eb-937b-40b73feb9d63 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:35.211914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:35.211914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:35.211914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:35.211914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:35.211914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:35.211914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:35.211914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:35.211914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:35.211914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:35.211914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:35.211914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:35.211914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:35.211914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:35.211914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:35.213108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:35.213108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:35.213108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:35.213108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:35.213108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:35.213108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:35.213108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:35.213108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:35.213108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:35.213108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:35.213108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:35.213108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:35.213108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:35.213108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:35.213108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:35.213108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:35.213108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:35.214277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:35.214277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:35.214277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:35.214277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:35.218903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 316/624] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 382 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:35.225164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b099de-ce94-40eb-937b-40b73feb9d63 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:35.299641 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801f56c1-427b-4baa-8e4c-423e88fbc307 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:35.337755 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab7061-6976-4154-a6ca-fdfcef590d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:35.338944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab7061-6976-4154-a6ca-fdfcef590d98 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:35.339412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab7061-6976-4154-a6ca-fdfcef590d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:35.339892 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ab7061-6976-4154-a6ca-fdfcef590d98 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:35.377992 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:35.417758 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab7061-6976-4154-a6ca-fdfcef590d98 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:35.419577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6ab7061-6976-4154-a6ca-fdfcef590d98 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:35.449052 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:35.458456 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b099de-ce94-40eb-937b-40b73feb9d63 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:35.532408 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8464300-1faa-49cb-bb03-8f8cd23d86f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:35.534386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8464300-1faa-49cb-bb03-8f8cd23d86f9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:35.535596 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8464300-1faa-49cb-bb03-8f8cd23d86f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:35.536098 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8464300-1faa-49cb-bb03-8f8cd23d86f9 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:35.538589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8464300-1faa-49cb-bb03-8f8cd23d86f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:35.540087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8464300-1faa-49cb-bb03-8f8cd23d86f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:35.542838 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8464300-1faa-49cb-bb03-8f8cd23d86f9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:35.547043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 309/625] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:35 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:54:35.557106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab7061-6976-4154-a6ca-fdfcef590d98 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:35.571569 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 317/626] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 347 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:35.577113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8464300-1faa-49cb-bb03-8f8cd23d86f9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:35.634928 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:35.672759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6ab7061-6976-4154-a6ca-fdfcef590d98 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:35.682678 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8464300-1faa-49cb-bb03-8f8cd23d86f9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:35.709212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8607a8d2-2244-40f3-9b83-d1d3025704d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:35.710044 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607a8d2-2244-40f3-9b83-d1d3025704d6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:35.710331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607a8d2-2244-40f3-9b83-d1d3025704d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:35.710611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8607a8d2-2244-40f3-9b83-d1d3025704d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:35.717468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48eeee5c-2b9b-4c1d-9f29-1b3e5d086a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:35.718283 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eeee5c-2b9b-4c1d-9f29-1b3e5d086a71 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:35.718572 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eeee5c-2b9b-4c1d-9f29-1b3e5d086a71 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:35.718853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48eeee5c-2b9b-4c1d-9f29-1b3e5d086a71 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:35.721754 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607a8d2-2244-40f3-9b83-d1d3025704d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:35.723244 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8607a8d2-2244-40f3-9b83-d1d3025704d6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:35.725753 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eeee5c-2b9b-4c1d-9f29-1b3e5d086a71 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:35.727216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48eeee5c-2b9b-4c1d-9f29-1b3e5d086a71 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:35.740770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-8607a8d2-2244-40f3-9b83-d1d3025704d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:35.740770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:35.740770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:35.740770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:35.740770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:35.740770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:35.740770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:35.740770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:35.740770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:35.740770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:35.740770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:35.740770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:35.740770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:35.740770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:35.740770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:35.741695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:35.741695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:35.741695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:35.741695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:35.741695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:35.741695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:35.741695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:35.741695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:35.741695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:35.741695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:35.741695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:35.741695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:35.741695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:35.741695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:35.741695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:35.741695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:35.741695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:35.750697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:35.750697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:35.750697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:35.750697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:35.750989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-48eeee5c-2b9b-4c1d-9f29-1b3e5d086a71 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:35.750989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:35.750989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:35.750989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:35.750989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:35.750989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:35.750989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:35.750989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:35.750989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:35.750989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:35.750989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:35.750989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:35.750989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:35.750989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:35.750989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:35.752039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:35.752039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:35.752039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:35.752039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:35.752039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:35.752039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:35.752039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:35.752039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:35.752039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:35.752039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:35.752039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:35.752039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:35.752039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:35.752039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:35.752039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:35.752039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:35.752039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:35.759468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:35.759468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:35.759468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:35.759468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:35.759468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:35.759468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:35.759468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:35.759468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:35.760057 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 310/627] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:54:35.760149 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 318/628] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:54:35.760235 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361b7d1-ae6f-426b-b2f8-3dcd3cad1ed3 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:35.760235 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361b7d1-ae6f-426b-b2f8-3dcd3cad1ed3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:35.760235 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8361b7d1-ae6f-426b-b2f8-3dcd3cad1ed3 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:35.760452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b5799-36de-48b4-99e6-44bd22f08774 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:35.760452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b5799-36de-48b4-99e6-44bd22f08774 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:35.760452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1b5799-36de-48b4-99e6-44bd22f08774 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:35.760452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 319/629] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:35.760744 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 311/630] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:35 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:35.760827 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa97ffb-8742-4647-8c55-043f07ff1cbb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:35.760827 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa97ffb-8742-4647-8c55-043f07ff1cbb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:35.760827 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faa97ffb-8742-4647-8c55-043f07ff1cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:35.761056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361b7d1-ae6f-426b-b2f8-3dcd3cad1ed3 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:35.775257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-faa97ffb-8742-4647-8c55-043f07ff1cbb None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:35.776480 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-faa97ffb-8742-4647-8c55-043f07ff1cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:35.792667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:35.853122 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 320/631] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:54:35.854130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa97ffb-8742-4647-8c55-043f07ff1cbb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:35.880717 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8361b7d1-ae6f-426b-b2f8-3dcd3cad1ed3 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:35.902950 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:35.918318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb652e89-4001-4342-895d-8d8cb46e68fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:35.919048 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb652e89-4001-4342-895d-8d8cb46e68fd None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:35.919339 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb652e89-4001-4342-895d-8d8cb46e68fd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:35.919618 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb652e89-4001-4342-895d-8d8cb46e68fd None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:35.920384 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb652e89-4001-4342-895d-8d8cb46e68fd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:35.920699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb652e89-4001-4342-895d-8d8cb46e68fd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:35.922104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb652e89-4001-4342-895d-8d8cb46e68fd None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:35.934063 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 312/632] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:35.935356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fe5b8-6aff-4c82-9c9c-9e1e9d735447 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:35.935652 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fe5b8-6aff-4c82-9c9c-9e1e9d735447 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:35.935965 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8fe5b8-6aff-4c82-9c9c-9e1e9d735447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:35.953491 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-fc8fe5b8-6aff-4c82-9c9c-9e1e9d735447 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:35.954834 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-fc8fe5b8-6aff-4c82-9c9c-9e1e9d735447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:35.965351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faa97ffb-8742-4647-8c55-043f07ff1cbb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:36.001144 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f158f-f513-4b9f-b70b-ac06beaf560b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:36.001818 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f158f-f513-4b9f-b70b-ac06beaf560b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:36.002105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f158f-f513-4b9f-b70b-ac06beaf560b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:36.002387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b56f158f-f513-4b9f-b70b-ac06beaf560b None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:36.003302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f158f-f513-4b9f-b70b-ac06beaf560b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:36.003624 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f158f-f513-4b9f-b70b-ac06beaf560b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:36.005211 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b56f158f-f513-4b9f-b70b-ac06beaf560b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:36.022420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 321/633] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:36.022420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f158f-f513-4b9f-b70b-ac06beaf560b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:36.030544 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 313/634] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:36.033250 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:36.033843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fe5b8-6aff-4c82-9c9c-9e1e9d735447 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:36.133304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b56f158f-f513-4b9f-b70b-ac06beaf560b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:36.139805 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8fe5b8-6aff-4c82-9c9c-9e1e9d735447 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:36.151658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:36.169182 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfcae1-22f3-4a15-9ffd-4dd30145b6f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:36.169997 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfcae1-22f3-4a15-9ffd-4dd30145b6f2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:36.170291 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfcae1-22f3-4a15-9ffd-4dd30145b6f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:36.170597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11dfcae1-22f3-4a15-9ffd-4dd30145b6f2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:36.177284 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8e7a3-4146-4782-a275-ea47c7ebdeed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:36.178388 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8e7a3-4146-4782-a275-ea47c7ebdeed None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:36.178921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8e7a3-4146-4782-a275-ea47c7ebdeed None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:36.179399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd8e7a3-4146-4782-a275-ea47c7ebdeed None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:36.181500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11dfcae1-22f3-4a15-9ffd-4dd30145b6f2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:36.186788 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8e7a3-4146-4782-a275-ea47c7ebdeed None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:36.188638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dd8e7a3-4146-4782-a275-ea47c7ebdeed None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:36.189450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-11dfcae1-22f3-4a15-9ffd-4dd30145b6f2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:36.189450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:36.189450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:36.189450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:36.189450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:36.189450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:36.189450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:36.189450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:36.189450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:36.189450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:36.189450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:36.189450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:36.189450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:36.189450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:36.189450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:36.189450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:36.190371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:36.190371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:36.190371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:36.190371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:36.190371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:36.190371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:36.190371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:36.190371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:36.190371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:36.190371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:36.190371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:36.190371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:36.190371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:36.190371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:36.190371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:36.190371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:36.191242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:36.191242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:36.191242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:36.191242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:36.191242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:36.191242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:36.191242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:36.191242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:36.191242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:36.191242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:36.191242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:36.191242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:36.191242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 322/635] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:36.191989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfcae1-22f3-4a15-9ffd-4dd30145b6f2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:36.209089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-6dd8e7a3-4146-4782-a275-ea47c7ebdeed None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:36.209089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:36.209089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:36.209089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:36.209089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:36.209089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:36.209089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:36.209089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:36.209089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:36.209089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:36.209089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:36.209089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:36.209089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:36.209089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:36.209089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:36.210104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:36.210104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:36.210104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:36.210104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:36.210104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:36.210104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:36.210104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:36.210104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:36.210104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:36.210104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:36.210104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:36.210104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:36.210104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:36.210104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:36.210104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:36.210104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:36.210104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:36.211015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:36.211015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:36.211015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:36.211015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:36.211015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:36.211015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:36.211015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:36.211015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:36.215573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 314/636] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:36.216563 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8e7a3-4146-4782-a275-ea47c7ebdeed None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:36.272343 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:36.306315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11dfcae1-22f3-4a15-9ffd-4dd30145b6f2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:36.329173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dd8e7a3-4146-4782-a275-ea47c7ebdeed None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:36.342060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebdea1-839b-4ddc-a54d-ff332a4c7081 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:36.342981 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebdea1-839b-4ddc-a54d-ff332a4c7081 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:36.343275 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebdea1-839b-4ddc-a54d-ff332a4c7081 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:36.343557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7ebdea1-839b-4ddc-a54d-ff332a4c7081 None admin] PATH_INFO: `/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/77fda27641c84fccb68a5cdaf4e8dc4b/roles/83bf7d9c29d4464dab8030f6d706cd6d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:36.358774 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebdea1-839b-4ddc-a54d-ff332a4c7081 None admin] RBAC: Authorizing `identity:create_grant(project_id=5584ebdaf133444ca69efc5554bf48e9, user_id=77fda27641c84fccb68a5cdaf4e8dc4b, role_id=83bf7d9c29d4464dab8030f6d706cd6d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:36.360187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7ebdea1-839b-4ddc-a54d-ff332a4c7081 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:36.368058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e7093-9308-44b8-a006-758b5107e41f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:36.368874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e7093-9308-44b8-a006-758b5107e41f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:36.369169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e7093-9308-44b8-a006-758b5107e41f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:36.369663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55e7093-9308-44b8-a006-758b5107e41f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:36.384058 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 323/637] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:54:36 2022] PUT /identity/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/77fda27641c84fccb68a5cdaf4e8dc4b/roles/83bf7d9c29d4464dab8030f6d706cd6d => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:54:36.384058 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebdea1-839b-4ddc-a54d-ff332a4c7081 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:36.391204 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55e7093-9308-44b8-a006-758b5107e41f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:36.399989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-d55e7093-9308-44b8-a006-758b5107e41f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:36.399989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:36.399989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:36.399989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:36.399989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:36.399989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:36.399989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:36.399989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:36.399989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:36.399989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:36.399989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:36.399989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:36.399989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:36.399989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:36.399989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:36.399989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:36.401021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:36.401021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:36.401021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:36.401021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:36.401021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:36.401021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:36.401021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:36.401021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:36.401021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:36.401021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:36.401021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:36.401021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:36.401021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:36.401021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:36.401021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:36.401021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:36.401924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:36.401924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:36.401924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:36.401924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:36.401924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:36.401924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:36.401924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:36.401924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:36.401924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:36.401924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:36.401924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:36.401924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:36.401924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 315/638] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:36.402628 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e7093-9308-44b8-a006-758b5107e41f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:36.435581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:36.502426 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7ebdea1-839b-4ddc-a54d-ff332a4c7081 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:36.519078 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55e7093-9308-44b8-a006-758b5107e41f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:36.524907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:36.541756 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0af1a82a-f86d-4903-8864-32134538c97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:36.542453 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1a82a-f86d-4903-8864-32134538c97c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:36.542744 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1a82a-f86d-4903-8864-32134538c97c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:36.543028 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0af1a82a-f86d-4903-8864-32134538c97c None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:36.543799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1a82a-f86d-4903-8864-32134538c97c None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:36.544208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1a82a-f86d-4903-8864-32134538c97c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:36.545962 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0af1a82a-f86d-4903-8864-32134538c97c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:36.557112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 324/639] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:36.558870 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74461a4a-8d34-4c7e-9f10-6dae355f518d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:36.559166 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74461a4a-8d34-4c7e-9f10-6dae355f518d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:36.559433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74461a4a-8d34-4c7e-9f10-6dae355f518d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:36.563910 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 325/640] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:36.563910 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74461a4a-8d34-4c7e-9f10-6dae355f518d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:36.570944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03495f3e-1920-4e7d-9008-02deac4a3f4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:36.571723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03495f3e-1920-4e7d-9008-02deac4a3f4f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:36.572055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03495f3e-1920-4e7d-9008-02deac4a3f4f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:36.572387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03495f3e-1920-4e7d-9008-02deac4a3f4f None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:36.573190 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03495f3e-1920-4e7d-9008-02deac4a3f4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:36.573506 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03495f3e-1920-4e7d-9008-02deac4a3f4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:36.574963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03495f3e-1920-4e7d-9008-02deac4a3f4f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:36.592466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 316/641] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:36.597386 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03495f3e-1920-4e7d-9008-02deac4a3f4f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:36.678894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74461a4a-8d34-4c7e-9f10-6dae355f518d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:36.694986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:36.705649 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03495f3e-1920-4e7d-9008-02deac4a3f4f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:36.717220 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ac5544-a3e7-45d4-91b4-29880ab34b01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:36.718300 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac5544-a3e7-45d4-91b4-29880ab34b01 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:36.718549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac5544-a3e7-45d4-91b4-29880ab34b01 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:36.718778 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ac5544-a3e7-45d4-91b4-29880ab34b01 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:36.719544 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ac5544-a3e7-45d4-91b4-29880ab34b01 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:36.719900 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ac5544-a3e7-45d4-91b4-29880ab34b01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:36.721311 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ac5544-a3e7-45d4-91b4-29880ab34b01 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:36.729981 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 326/642] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:36.731205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac0dbd-c96b-4a30-a271-aa77db17203e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:36.731566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac0dbd-c96b-4a30-a271-aa77db17203e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:36.731876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ac0dbd-c96b-4a30-a271-aa77db17203e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:36.748259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251530f-0d90-4185-9a59-b01f07bbf383 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:36.748920 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-32ac0dbd-c96b-4a30-a271-aa77db17203e None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:36.749953 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251530f-0d90-4185-9a59-b01f07bbf383 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:36.750133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-32ac0dbd-c96b-4a30-a271-aa77db17203e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:36.750643 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251530f-0d90-4185-9a59-b01f07bbf383 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:36.751097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251530f-0d90-4185-9a59-b01f07bbf383 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:36.765274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251530f-0d90-4185-9a59-b01f07bbf383 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:36.767678 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251530f-0d90-4185-9a59-b01f07bbf383 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:36.789724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-5251530f-0d90-4185-9a59-b01f07bbf383 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:36.789724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:36.789724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:36.789724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:36.789724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:36.789724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:36.789724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:36.789724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:36.789724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:36.789724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:36.789724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:36.789724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:36.789724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:36.789724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:36.789724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:36.790792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:36.790792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:36.790792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:36.790792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:36.790792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:36.790792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:36.790792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:36.790792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:36.790792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:36.790792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:36.790792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:36.790792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:36.790792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:36.790792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:36.790792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:36.790792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:36.790792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:36.791678 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:36.791678 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:36.791678 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:36.791678 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:36.794322 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:36.795932 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 317/643] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:36.796821 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251530f-0d90-4185-9a59-b01f07bbf383 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:36.824496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 327/644] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:36.825284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac0dbd-c96b-4a30-a271-aa77db17203e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:36.911704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251530f-0d90-4185-9a59-b01f07bbf383 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:36.929345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ac0dbd-c96b-4a30-a271-aa77db17203e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:36.947396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2ff1e-43f4-482b-a055-abc992a0d99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:36.948101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2ff1e-43f4-482b-a055-abc992a0d99b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:36.948392 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2ff1e-43f4-482b-a055-abc992a0d99b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:36.948673 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd2ff1e-43f4-482b-a055-abc992a0d99b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:36.949516 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2ff1e-43f4-482b-a055-abc992a0d99b None admin] RBAC: Adding query filter params (role.id=83bf7d9c29d4464dab8030f6d706cd6d, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=4751abc9f67947b3ad17ba2b78ce2c12) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:36.949836 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2ff1e-43f4-482b-a055-abc992a0d99b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:36.951347 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd2ff1e-43f4-482b-a055-abc992a0d99b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:36.957923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 318/645] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:54:36 2022] GET /identity/v3/role_assignments?user.id=4751abc9f67947b3ad17ba2b78ce2c12&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=83bf7d9c29d4464dab8030f6d706cd6d => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:36.960940 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2ff1e-43f4-482b-a055-abc992a0d99b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:36.970015 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb24b1-0a1f-465c-bcaf-d52b737d0a8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:36.971011 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb24b1-0a1f-465c-bcaf-d52b737d0a8c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:36.971398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb24b1-0a1f-465c-bcaf-d52b737d0a8c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:36.971774 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bb24b1-0a1f-465c-bcaf-d52b737d0a8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:36.974613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:36.987803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb24b1-0a1f-465c-bcaf-d52b737d0a8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:36.989515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bb24b1-0a1f-465c-bcaf-d52b737d0a8c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:37.010538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a1bb24b1-0a1f-465c-bcaf-d52b737d0a8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:37.010538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:37.010538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:37.010538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:37.010538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:37.010538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:37.010538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:37.010538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:37.010538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:37.010538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:37.010538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:37.010538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:37.010538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:37.010538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:37.010538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:37.011743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:37.011743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:37.011743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:37.011743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:37.011743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:37.011743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:37.011743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:37.011743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:37.011743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:37.011743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:37.011743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:37.011743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:37.011743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:37.011743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:37.011743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:37.011743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:37.011743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:37.013112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:37.013112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:37.013112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:37.013112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:37.014536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 328/646] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:37.015484 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb24b1-0a1f-465c-bcaf-d52b737d0a8c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:37.048883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:37.109203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd2ff1e-43f4-482b-a055-abc992a0d99b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:37.147580 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc05b83-3453-4ef9-bd76-eb2e0e7540f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:37.148544 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc05b83-3453-4ef9-bd76-eb2e0e7540f7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:37.148922 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc05b83-3453-4ef9-bd76-eb2e0e7540f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:37.149236 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc05b83-3453-4ef9-bd76-eb2e0e7540f7 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:37.150076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc05b83-3453-4ef9-bd76-eb2e0e7540f7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:37.150633 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc05b83-3453-4ef9-bd76-eb2e0e7540f7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:37.152162 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc05b83-3453-4ef9-bd76-eb2e0e7540f7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:37.155615 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bb24b1-0a1f-465c-bcaf-d52b737d0a8c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:37.166151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 319/647] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:37.167876 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc05b83-3453-4ef9-bd76-eb2e0e7540f7 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:37.201568 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a89bc-db83-4576-b409-d57215bbd83f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:37.202600 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a89bc-db83-4576-b409-d57215bbd83f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:37.202985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a89bc-db83-4576-b409-d57215bbd83f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:37.203369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a89bc-db83-4576-b409-d57215bbd83f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:37.218880 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a89bc-db83-4576-b409-d57215bbd83f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:37.230109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8f4a89bc-db83-4576-b409-d57215bbd83f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:37.230109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:37.230109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:37.230109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:37.230109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:37.230109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:37.230109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:37.230109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:37.230109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:37.230109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:37.230109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:37.230109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:37.230109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:37.230109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:37.230109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:37.230109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:37.231247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:37.231247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:37.231247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:37.231247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:37.231247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:37.231247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:37.231247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:37.231247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:37.231247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:37.231247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:37.231247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:37.231247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:37.231247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:37.231247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:37.231247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:37.231247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:37.232337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:37.232337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:37.232337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:37.232337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:37.232337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:37.232337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:37.232337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:37.232337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:37.232337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:37.232337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:37.232337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:37.232337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:37.239062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 329/648] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:37 2022] GET /identity/v3/roles/service => generated 85 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:37.239062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a89bc-db83-4576-b409-d57215bbd83f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:37.242030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:37.282886 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc05b83-3453-4ef9-bd76-eb2e0e7540f7 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:37.327763 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055edda9-4b1a-4799-9ad6-6c6e7602e591 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:37.328773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055edda9-4b1a-4799-9ad6-6c6e7602e591 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:37.329208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055edda9-4b1a-4799-9ad6-6c6e7602e591 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:37.329657 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055edda9-4b1a-4799-9ad6-6c6e7602e591 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:37.330531 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055edda9-4b1a-4799-9ad6-6c6e7602e591 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:37.331017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055edda9-4b1a-4799-9ad6-6c6e7602e591 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:37.332793 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055edda9-4b1a-4799-9ad6-6c6e7602e591 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:37.345206 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 320/649] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:37.345841 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055edda9-4b1a-4799-9ad6-6c6e7602e591 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:37.355400 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:37.379335 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a89bc-db83-4576-b409-d57215bbd83f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:37.423588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789db20-9330-4d8f-8297-4d14d8131cb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:37.424732 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789db20-9330-4d8f-8297-4d14d8131cb8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:37.425204 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789db20-9330-4d8f-8297-4d14d8131cb8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:37.425664 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9789db20-9330-4d8f-8297-4d14d8131cb8 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:37.433937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789db20-9330-4d8f-8297-4d14d8131cb8 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:37.435724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9789db20-9330-4d8f-8297-4d14d8131cb8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:37.452792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9789db20-9330-4d8f-8297-4d14d8131cb8 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. May 25 15:54:37.452792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:37.452792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:37.452792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:37.452792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:37.452792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:37.452792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:37.452792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:37.452792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:37.452792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:37.452792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:37.452792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:37.452792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:37.452792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:37.452792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:37.452792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:37.453768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:37.454724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:37.454724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:37.454724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:37.454724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:37.454724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:37.454724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:37.454724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:37.454724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:37.454724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. May 25 15:54:37.454724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:37.455795 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055edda9-4b1a-4799-9ad6-6c6e7602e591 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:37.457249 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 330/650] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:37 2022] GET /identity/v3/users/glance => generated 84 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:37.458569 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789db20-9330-4d8f-8297-4d14d8131cb8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:37.487849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:37.499348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e103504-057c-4ed0-a63e-88ac571cb35e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:37.500125 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e103504-057c-4ed0-a63e-88ac571cb35e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:37.500426 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e103504-057c-4ed0-a63e-88ac571cb35e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:37.500711 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e103504-057c-4ed0-a63e-88ac571cb35e None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:37.501502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e103504-057c-4ed0-a63e-88ac571cb35e None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:37.501823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e103504-057c-4ed0-a63e-88ac571cb35e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:37.503287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e103504-057c-4ed0-a63e-88ac571cb35e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:37.517549 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 321/651] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:37 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:37.518527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e103504-057c-4ed0-a63e-88ac571cb35e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:37.571262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9789db20-9330-4d8f-8297-4d14d8131cb8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:37.606569 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79485f00-de79-480b-91f0-3a144b6d7b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:37.607648 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79485f00-de79-480b-91f0-3a144b6d7b65 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:37.608128 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79485f00-de79-480b-91f0-3a144b6d7b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:37.608583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79485f00-de79-480b-91f0-3a144b6d7b65 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:37.617695 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79485f00-de79-480b-91f0-3a144b6d7b65 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:37.619202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79485f00-de79-480b-91f0-3a144b6d7b65 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:37.627144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e103504-057c-4ed0-a63e-88ac571cb35e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:37.634202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-79485f00-de79-480b-91f0-3a144b6d7b65 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 15:54:37.634202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:37.634202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:37.634202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:37.634202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:37.634202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:37.634202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:37.634202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:37.634202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:37.634202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:37.634202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:37.634202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:37.634202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:37.634202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:37.634202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:37.634202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:37.635282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:37.636307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:37.636307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:37.636307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:37.636307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:37.636307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:37.636307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:37.636307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:37.636307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:37.636307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 15:54:37.636307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:37.640258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 331/652] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:37 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:37.641128 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79485f00-de79-480b-91f0-3a144b6d7b65 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:37.650885 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:37.674537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f1a27-ed47-42ff-a556-9f0d99fa81d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:37.676228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f1a27-ed47-42ff-a556-9f0d99fa81d4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:37.676728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f1a27-ed47-42ff-a556-9f0d99fa81d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:37.677431 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f1a27-ed47-42ff-a556-9f0d99fa81d4 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:37.678467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f1a27-ed47-42ff-a556-9f0d99fa81d4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:37.678961 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f1a27-ed47-42ff-a556-9f0d99fa81d4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:37.681752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f1a27-ed47-42ff-a556-9f0d99fa81d4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:37.710177 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 322/653] 10.222.0.85 () {56 vars in 1137 bytes} [Wed May 25 15:54:37 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:37.710177 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f1a27-ed47-42ff-a556-9f0d99fa81d4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:37.753206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79485f00-de79-480b-91f0-3a144b6d7b65 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:37.765452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:37.792919 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab612c-f79a-4988-940a-0743b99115bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:37.794728 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab612c-f79a-4988-940a-0743b99115bb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:37.795117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab612c-f79a-4988-940a-0743b99115bb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:37.795500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39ab612c-f79a-4988-940a-0743b99115bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:37.810229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab612c-f79a-4988-940a-0743b99115bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:37.811878 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39ab612c-f79a-4988-940a-0743b99115bb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:37.825125 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f1a27-ed47-42ff-a556-9f0d99fa81d4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:37.835681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-39ab612c-f79a-4988-940a-0743b99115bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:37.835681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:37.835681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:37.835681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:37.835681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:37.835681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:37.835681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:37.835681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:37.835681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:37.835681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:37.835681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:37.835681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:37.835681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:37.835681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:37.835681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:37.836649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:37.836649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:37.836649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:37.836649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:37.836649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:37.836649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:37.836649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:37.836649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:37.836649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:37.836649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:37.836649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:37.836649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:37.836649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:37.836649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:37.836649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:37.836649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:37.836649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:37.837559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:37.837559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:37.837559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:37.837559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:37.837559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 332/654] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:37.842545 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab612c-f79a-4988-940a-0743b99115bb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:37.868630 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc960cd6-2b77-4d77-97e8-47e682f3dbcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:37.869612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc960cd6-2b77-4d77-97e8-47e682f3dbcf None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:37.870080 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc960cd6-2b77-4d77-97e8-47e682f3dbcf None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:37.870597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc960cd6-2b77-4d77-97e8-47e682f3dbcf None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:37.871699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc960cd6-2b77-4d77-97e8-47e682f3dbcf None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:37.872274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc960cd6-2b77-4d77-97e8-47e682f3dbcf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:37.873971 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc960cd6-2b77-4d77-97e8-47e682f3dbcf None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:37.892568 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 323/655] 10.222.0.85 () {56 vars in 1137 bytes} [Wed May 25 15:54:37 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:37.893670 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc960cd6-2b77-4d77-97e8-47e682f3dbcf None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:37.923320 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:37.984049 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39ab612c-f79a-4988-940a-0743b99115bb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:38.001848 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc960cd6-2b77-4d77-97e8-47e682f3dbcf None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:38.005270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:38.030349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce2a32-78d1-4fc7-bfe1-8ffeaaf67c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:38.031378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce2a32-78d1-4fc7-bfe1-8ffeaaf67c30 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:38.031736 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce2a32-78d1-4fc7-bfe1-8ffeaaf67c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:38.032116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ce2a32-78d1-4fc7-bfe1-8ffeaaf67c30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:38.043613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343ad37-6a70-43ed-9af6-55dfa21d1e60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:38.044376 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343ad37-6a70-43ed-9af6-55dfa21d1e60 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:38.044670 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343ad37-6a70-43ed-9af6-55dfa21d1e60 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:38.044956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1343ad37-6a70-43ed-9af6-55dfa21d1e60 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:38.046155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343ad37-6a70-43ed-9af6-55dfa21d1e60 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:38.046477 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343ad37-6a70-43ed-9af6-55dfa21d1e60 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:38.048334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1343ad37-6a70-43ed-9af6-55dfa21d1e60 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:38.068647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 324/656] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:38.068647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343ad37-6a70-43ed-9af6-55dfa21d1e60 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:38.111924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce2a32-78d1-4fc7-bfe1-8ffeaaf67c30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:38.113751 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ce2a32-78d1-4fc7-bfe1-8ffeaaf67c30 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:38.180539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1343ad37-6a70-43ed-9af6-55dfa21d1e60 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:38.191729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:38.231036 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3c3e9-e07b-4c88-bee8-21f3e33fb30d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:38.231951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3c3e9-e07b-4c88-bee8-21f3e33fb30d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:38.232248 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3c3e9-e07b-4c88-bee8-21f3e33fb30d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:38.232529 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f3c3e9-e07b-4c88-bee8-21f3e33fb30d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:38.247434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 333/657] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:54:38.248836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce2a32-78d1-4fc7-bfe1-8ffeaaf67c30 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:38.272733 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:38.323485 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3c3e9-e07b-4c88-bee8-21f3e33fb30d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:38.325590 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f3c3e9-e07b-4c88-bee8-21f3e33fb30d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:38.363157 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ce2a32-78d1-4fc7-bfe1-8ffeaaf67c30 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:38.405232 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12169061-da04-4511-98b4-980b19036e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:38.406358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12169061-da04-4511-98b4-980b19036e3c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:38.406656 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12169061-da04-4511-98b4-980b19036e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:38.406944 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12169061-da04-4511-98b4-980b19036e3c None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:38.417882 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12169061-da04-4511-98b4-980b19036e3c None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:38.419447 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12169061-da04-4511-98b4-980b19036e3c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:38.440202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-12169061-da04-4511-98b4-980b19036e3c None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. May 25 15:54:38.440202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:38.440202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:38.440202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:38.440202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:38.440202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:38.440202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:38.440202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:38.440202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:38.440202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:38.440202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:38.440202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:38.440202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:38.440202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:38.440202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:38.440202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:38.441246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:38.442205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:38.442205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:38.442205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:38.442205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:38.442205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:38.442205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:38.442205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:38.442205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:38.442205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. May 25 15:54:38.442205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:38.442205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 334/658] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:38 2022] GET /identity/v3/users/neutron => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:38.442838 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12169061-da04-4511-98b4-980b19036e3c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:38.456943 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 325/659] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:54:38.458531 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3c3e9-e07b-4c88-bee8-21f3e33fb30d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:38.471052 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:38.536233 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:38.567117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12169061-da04-4511-98b4-980b19036e3c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:38.586075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f3c3e9-e07b-4c88-bee8-21f3e33fb30d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:38.605986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a6b7d-e4ea-4277-9403-34f0a36667d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:38.606823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a6b7d-e4ea-4277-9403-34f0a36667d7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:38.607113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a6b7d-e4ea-4277-9403-34f0a36667d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:38.607396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0a6b7d-e4ea-4277-9403-34f0a36667d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:38.617892 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a6b7d-e4ea-4277-9403-34f0a36667d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:38.619371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d0a6b7d-e4ea-4277-9403-34f0a36667d7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:38.621942 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b80898-e0cb-4d5d-99fe-42fdce35c01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:38.622614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b80898-e0cb-4d5d-99fe-42fdce35c01d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:38.622900 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b80898-e0cb-4d5d-99fe-42fdce35c01d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:38.623175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b80898-e0cb-4d5d-99fe-42fdce35c01d None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:38.624029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b80898-e0cb-4d5d-99fe-42fdce35c01d None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:38.624344 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b80898-e0cb-4d5d-99fe-42fdce35c01d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:38.625752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b80898-e0cb-4d5d-99fe-42fdce35c01d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:38.637965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9d0a6b7d-e4ea-4277-9403-34f0a36667d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:38.637965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:38.637965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:38.637965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:38.637965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:38.637965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:38.637965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:38.637965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:38.637965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:38.637965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:38.637965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:38.637965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:38.637965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:38.637965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:38.637965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:38.638884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:38.638884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:38.638884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:38.638884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:38.638884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:38.638884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:38.638884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:38.638884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:38.638884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:38.638884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:38.638884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:38.638884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:38.638884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:38.638884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:38.638884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:38.638884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:38.638884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:38.644520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:38.644520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:38.644520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:38.644520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:38.644520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 335/660] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:38.644520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a6b7d-e4ea-4277-9403-34f0a36667d7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:38.648972 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 326/661] 10.222.0.85 () {56 vars in 1139 bytes} [Wed May 25 15:54:38 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:38.653476 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebd1d0-d193-4a5d-8631-4dc1e8ea9f44 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:38.653476 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebd1d0-d193-4a5d-8631-4dc1e8ea9f44 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:38.653476 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ebd1d0-d193-4a5d-8631-4dc1e8ea9f44 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:38.656163 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 327/662] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:38 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:38.656990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebd1d0-d193-4a5d-8631-4dc1e8ea9f44 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:38.719309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:38.751845 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d0a6b7d-e4ea-4277-9403-34f0a36667d7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:38.762405 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ebd1d0-d193-4a5d-8631-4dc1e8ea9f44 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:38.788570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:38.791013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540a42ca-d8b3-4f23-bb80-9e87e881960c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:38.791889 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a42ca-d8b3-4f23-bb80-9e87e881960c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:38.792136 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a42ca-d8b3-4f23-bb80-9e87e881960c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:38.792468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540a42ca-d8b3-4f23-bb80-9e87e881960c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:38.803284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a42ca-d8b3-4f23-bb80-9e87e881960c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:38.804876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540a42ca-d8b3-4f23-bb80-9e87e881960c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:38.805060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c755c-3b75-42a6-8dac-d6e4fd01d8a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:38.805722 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c755c-3b75-42a6-8dac-d6e4fd01d8a3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:38.806014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c755c-3b75-42a6-8dac-d6e4fd01d8a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:38.806299 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4c755c-3b75-42a6-8dac-d6e4fd01d8a3 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:38.807075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c755c-3b75-42a6-8dac-d6e4fd01d8a3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:38.807772 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c755c-3b75-42a6-8dac-d6e4fd01d8a3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:38.808798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4c755c-3b75-42a6-8dac-d6e4fd01d8a3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:38.824925 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 328/663] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:38.824925 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c755c-3b75-42a6-8dac-d6e4fd01d8a3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:38.825321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-540a42ca-d8b3-4f23-bb80-9e87e881960c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:38.825321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:38.825321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:38.825321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:38.825321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:38.825321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:38.825321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:38.825321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:38.825321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:38.825321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:38.825321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:38.825321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:38.825321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:38.825321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:38.825321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:38.826131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:38.826131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:38.826131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:38.826131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:38.826131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:38.826131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:38.826131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:38.826131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:38.826131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:38.826131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:38.826131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:38.826131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:38.826131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:38.826131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:38.826131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:38.826131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:38.826131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:38.827059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:38.827059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:38.827059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:38.827059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:38.827059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 336/664] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:38.827059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f888566-ddef-42b3-a34e-6beff898e669 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:38.827059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f888566-ddef-42b3-a34e-6beff898e669 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:38.827059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f888566-ddef-42b3-a34e-6beff898e669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:38.847125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4f888566-ddef-42b3-a34e-6beff898e669 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:38.848515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4f888566-ddef-42b3-a34e-6beff898e669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:38.930662 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4c755c-3b75-42a6-8dac-d6e4fd01d8a3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:38.932301 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 337/665] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:38.933249 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f888566-ddef-42b3-a34e-6beff898e669 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:38.969253 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0eca2d-d33d-46a3-b5b5-8fff41500718 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:38.969944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0eca2d-d33d-46a3-b5b5-8fff41500718 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:38.970210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0eca2d-d33d-46a3-b5b5-8fff41500718 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:38.970489 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a0eca2d-d33d-46a3-b5b5-8fff41500718 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:38.990286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:39.023847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:39.038355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f888566-ddef-42b3-a34e-6beff898e669 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:39.044594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0eca2d-d33d-46a3-b5b5-8fff41500718 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:39.046000 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a0eca2d-d33d-46a3-b5b5-8fff41500718 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:39.073658 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aebb5e-57ed-47fc-9835-d8d8a1f25c31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:39.074461 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aebb5e-57ed-47fc-9835-d8d8a1f25c31 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:39.074749 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aebb5e-57ed-47fc-9835-d8d8a1f25c31 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:39.075055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81aebb5e-57ed-47fc-9835-d8d8a1f25c31 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:39.081889 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aebb5e-57ed-47fc-9835-d8d8a1f25c31 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:39.083700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81aebb5e-57ed-47fc-9835-d8d8a1f25c31 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:39.097363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-81aebb5e-57ed-47fc-9835-d8d8a1f25c31 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:39.097363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:39.097363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:39.097363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:39.097363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:39.097363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:39.097363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:39.097363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:39.097363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:39.097363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:39.097363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:39.097363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:39.097363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:39.097363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:39.097363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:39.098253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:39.098253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:39.098253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:39.098253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:39.098253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:39.098253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:39.098253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:39.098253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:39.098253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:39.098253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:39.098253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:39.098253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:39.098253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:39.098253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:39.098253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:39.098253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:39.098253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:39.099160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:39.099160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:39.099160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:39.099160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:39.099160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:39.099160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:39.099160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:39.099160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:39.099160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 338/666] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:39.100705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aebb5e-57ed-47fc-9835-d8d8a1f25c31 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:39.150894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 329/667] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:54:39.152318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0eca2d-d33d-46a3-b5b5-8fff41500718 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:39.218870 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81aebb5e-57ed-47fc-9835-d8d8a1f25c31 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:39.232171 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:39.246176 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:39.255793 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d8c49-2733-482e-a5f8-f6fd30911e36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:39.256431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8c49-2733-482e-a5f8-f6fd30911e36 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:39.256727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8c49-2733-482e-a5f8-f6fd30911e36 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:39.257153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c56d8c49-2733-482e-a5f8-f6fd30911e36 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:39.257862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d8c49-2733-482e-a5f8-f6fd30911e36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:39.258089 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d8c49-2733-482e-a5f8-f6fd30911e36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:39.259554 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c56d8c49-2733-482e-a5f8-f6fd30911e36 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:39.265695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a0eca2d-d33d-46a3-b5b5-8fff41500718 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:39.277379 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 339/668] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:39.277379 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d8c49-2733-482e-a5f8-f6fd30911e36 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:39.312121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf19553-01e2-4aaf-978d-d345ce3202d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:39.312949 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf19553-01e2-4aaf-978d-d345ce3202d8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:39.313241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf19553-01e2-4aaf-978d-d345ce3202d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:39.313524 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf19553-01e2-4aaf-978d-d345ce3202d8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:39.324042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf19553-01e2-4aaf-978d-d345ce3202d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:39.331846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-ebf19553-01e2-4aaf-978d-d345ce3202d8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:39.331846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:39.331846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:39.331846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:39.331846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:39.331846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:39.331846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:39.331846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:39.331846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:39.331846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:39.331846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:39.331846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:39.331846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:39.331846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:39.331846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:39.331846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:39.333156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:39.333156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:39.333156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:39.333156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:39.333156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:39.333156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:39.333156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:39.333156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:39.333156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:39.333156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:39.333156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:39.333156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:39.333156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:39.333156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:39.333156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:39.333156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:39.334014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:39.334014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:39.334014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:39.334014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:39.334014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:39.334014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:39.334014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:39.334014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:39.334014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:39.334014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:39.334014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:39.334014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:39.338300 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 330/669] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:39.338953 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf19553-01e2-4aaf-978d-d345ce3202d8 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:39.387621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c56d8c49-2733-482e-a5f8-f6fd30911e36 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:39.421979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185df711-3bc1-4855-8f58-772720e9366f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:39.422651 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185df711-3bc1-4855-8f58-772720e9366f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:39.422941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185df711-3bc1-4855-8f58-772720e9366f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:39.423219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-185df711-3bc1-4855-8f58-772720e9366f None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:39.424045 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185df711-3bc1-4855-8f58-772720e9366f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:39.424363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185df711-3bc1-4855-8f58-772720e9366f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:39.425838 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-185df711-3bc1-4855-8f58-772720e9366f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:39.437271 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 340/670] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:39.437409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf19553-01e2-4aaf-978d-d345ce3202d8 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:39.438047 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185df711-3bc1-4855-8f58-772720e9366f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:39.469439 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:39.476310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab36b15-6782-4d9f-a663-67b3a09a25bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:39.477119 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab36b15-6782-4d9f-a663-67b3a09a25bd None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:39.477406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab36b15-6782-4d9f-a663-67b3a09a25bd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:39.477687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab36b15-6782-4d9f-a663-67b3a09a25bd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:39.488631 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab36b15-6782-4d9f-a663-67b3a09a25bd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:39.490106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab36b15-6782-4d9f-a663-67b3a09a25bd None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:39.491814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:39.508444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-7ab36b15-6782-4d9f-a663-67b3a09a25bd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:39.508444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:39.508444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:39.508444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:39.508444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:39.508444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:39.508444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:39.508444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:39.508444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:39.508444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:39.508444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:39.508444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:39.508444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:39.508444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:39.508444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:39.509341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:39.509341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:39.509341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:39.509341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:39.509341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:39.509341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:39.509341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:39.509341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:39.509341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:39.509341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:39.509341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:39.509341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:39.509341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:39.509341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:39.509341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:39.509341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:39.509341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:39.510228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:39.510228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:39.510228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:39.510228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:39.510228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 331/671] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:39.510514 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab36b15-6782-4d9f-a663-67b3a09a25bd None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:39.541629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-185df711-3bc1-4855-8f58-772720e9366f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:39.575706 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b711fee7-57d9-4fee-a414-675783ada05b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:39.576465 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711fee7-57d9-4fee-a414-675783ada05b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:39.576754 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711fee7-57d9-4fee-a414-675783ada05b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:39.577032 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b711fee7-57d9-4fee-a414-675783ada05b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:39.584070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711fee7-57d9-4fee-a414-675783ada05b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:39.585527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b711fee7-57d9-4fee-a414-675783ada05b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:39.599640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b711fee7-57d9-4fee-a414-675783ada05b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:39.599640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:39.599640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:39.599640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:39.599640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:39.599640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:39.599640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:39.599640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:39.599640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:39.599640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:39.599640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:39.599640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:39.599640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:39.599640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:39.599640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:39.600836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:39.600836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:39.600836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:39.600836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:39.600836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:39.600836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:39.600836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:39.600836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:39.600836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:39.600836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:39.600836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:39.600836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:39.600836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:39.600836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:39.600836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:39.600836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:39.600836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:39.601759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:39.601759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:39.601759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:39.601759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:39.601759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:39.601759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:39.601759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:39.601759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:39.601759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 341/672] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:39.602241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48388539-90f7-486d-b14e-3354bf4feb69 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:39.602718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48388539-90f7-486d-b14e-3354bf4feb69 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:39.602718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48388539-90f7-486d-b14e-3354bf4feb69 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:39.603990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 342/673] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:39.607983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48388539-90f7-486d-b14e-3354bf4feb69 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:39.618080 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab36b15-6782-4d9f-a663-67b3a09a25bd None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:39.652962 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218bd8d3-dda0-49c0-8479-ad603b8e61c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:39.653653 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218bd8d3-dda0-49c0-8479-ad603b8e61c2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:39.653931 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218bd8d3-dda0-49c0-8479-ad603b8e61c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:39.654205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218bd8d3-dda0-49c0-8479-ad603b8e61c2 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:39.654962 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218bd8d3-dda0-49c0-8479-ad603b8e61c2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:39.655279 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218bd8d3-dda0-49c0-8479-ad603b8e61c2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:39.656689 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218bd8d3-dda0-49c0-8479-ad603b8e61c2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:39.664540 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 332/674] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:39.665316 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218bd8d3-dda0-49c0-8479-ad603b8e61c2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:39.696418 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:39.706662 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48388539-90f7-486d-b14e-3354bf4feb69 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:39.721272 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:39.740490 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c0e5e-12b6-4bd6-bdd9-5a8cd357a5e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:39.741154 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c0e5e-12b6-4bd6-bdd9-5a8cd357a5e1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:39.741450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c0e5e-12b6-4bd6-bdd9-5a8cd357a5e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:39.741731 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641c0e5e-12b6-4bd6-bdd9-5a8cd357a5e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:39.742580 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c0e5e-12b6-4bd6-bdd9-5a8cd357a5e1 None admin] RBAC: Adding query filter params (role.id=83bf7d9c29d4464dab8030f6d706cd6d, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=196dc406a95444c39babc50d59f9eb6b) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:39.742906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c0e5e-12b6-4bd6-bdd9-5a8cd357a5e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:39.744401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641c0e5e-12b6-4bd6-bdd9-5a8cd357a5e1 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:39.750591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 343/675] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:54:39 2022] GET /identity/v3/role_assignments?user.id=196dc406a95444c39babc50d59f9eb6b&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=83bf7d9c29d4464dab8030f6d706cd6d => generated 608 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:39.751364 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c0e5e-12b6-4bd6-bdd9-5a8cd357a5e1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:39.766006 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218bd8d3-dda0-49c0-8479-ad603b8e61c2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:39.805484 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a31c25f-2533-4c47-bdb6-f9d4a01945b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:39.806205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31c25f-2533-4c47-bdb6-f9d4a01945b1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:39.806476 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31c25f-2533-4c47-bdb6-f9d4a01945b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:39.806758 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a31c25f-2533-4c47-bdb6-f9d4a01945b1 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:39.807930 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31c25f-2533-4c47-bdb6-f9d4a01945b1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:39.808253 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31c25f-2533-4c47-bdb6-f9d4a01945b1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:39.810091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a31c25f-2533-4c47-bdb6-f9d4a01945b1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:39.828442 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 333/676] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:39.828442 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5d997-ed4e-4636-9c6a-3fc5371d5c67 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:39.828442 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5d997-ed4e-4636-9c6a-3fc5371d5c67 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:39.828442 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c5d997-ed4e-4636-9c6a-3fc5371d5c67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:39.844846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-35c5d997-ed4e-4636-9c6a-3fc5371d5c67 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:39.846072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-35c5d997-ed4e-4636-9c6a-3fc5371d5c67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:39.875896 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641c0e5e-12b6-4bd6-bdd9-5a8cd357a5e1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:39.912535 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4888c1-3a65-4fe0-9d90-5bb70d0d87bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:39.913641 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4888c1-3a65-4fe0-9d90-5bb70d0d87bb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:39.914138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4888c1-3a65-4fe0-9d90-5bb70d0d87bb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:39.914620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd4888c1-3a65-4fe0-9d90-5bb70d0d87bb None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:39.915643 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4888c1-3a65-4fe0-9d90-5bb70d0d87bb None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:39.916220 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4888c1-3a65-4fe0-9d90-5bb70d0d87bb None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:39.918162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd4888c1-3a65-4fe0-9d90-5bb70d0d87bb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:39.920916 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 334/677] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:39.922391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5d997-ed4e-4636-9c6a-3fc5371d5c67 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:39.933600 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 344/678] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:39.934969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4888c1-3a65-4fe0-9d90-5bb70d0d87bb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:39.958562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:39.984611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:40.046693 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c5d997-ed4e-4636-9c6a-3fc5371d5c67 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:40.059627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd4888c1-3a65-4fe0-9d90-5bb70d0d87bb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:40.084037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81b161-6b30-4265-a3f2-01bc386a62b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:40.084777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81b161-6b30-4265-a3f2-01bc386a62b4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:40.085060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81b161-6b30-4265-a3f2-01bc386a62b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:40.085334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad81b161-6b30-4265-a3f2-01bc386a62b4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:40.095843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81b161-6b30-4265-a3f2-01bc386a62b4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:40.097124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac8f48-bd85-4cbb-98df-0ca7b0184701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:40.097323 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad81b161-6b30-4265-a3f2-01bc386a62b4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:40.098290 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac8f48-bd85-4cbb-98df-0ca7b0184701 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:40.098751 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac8f48-bd85-4cbb-98df-0ca7b0184701 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:40.099248 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac8f48-bd85-4cbb-98df-0ca7b0184701 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:40.106530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac8f48-bd85-4cbb-98df-0ca7b0184701 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:40.108250 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac8f48-bd85-4cbb-98df-0ca7b0184701 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:40.114883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-ad81b161-6b30-4265-a3f2-01bc386a62b4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:40.114883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:40.114883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:40.114883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:40.114883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:40.114883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:40.114883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:40.114883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:40.114883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:40.114883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:40.114883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:40.114883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:40.114883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:40.114883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:40.114883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:40.115750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:40.115750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:40.115750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:40.115750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:40.115750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:40.115750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:40.115750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:40.115750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:40.115750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:40.115750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:40.115750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:40.115750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:40.115750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:40.115750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:40.115750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:40.115750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:40.115750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:40.116715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:40.116715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:40.116715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:40.116715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:40.116715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 335/679] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:40.120650 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81b161-6b30-4265-a3f2-01bc386a62b4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:40.125551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-37ac8f48-bd85-4cbb-98df-0ca7b0184701 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:40.125551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:40.125551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:40.125551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:40.125551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:40.125551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:40.125551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:40.125551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:40.125551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:40.125551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:40.125551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:40.125551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:40.125551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:40.125551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:40.125551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:40.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:40.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:40.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:40.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:40.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:40.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:40.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:40.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:40.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:40.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:40.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:40.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:40.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:40.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:40.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:40.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:40.126424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:40.127353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:40.127353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:40.127353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:40.127353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:40.127353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:40.127353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:40.127353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:40.127353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:40.127353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 345/680] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:40.128342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac8f48-bd85-4cbb-98df-0ca7b0184701 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:40.207991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:40.227776 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:40.232400 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad81b161-6b30-4265-a3f2-01bc386a62b4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:40.233680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac8f48-bd85-4cbb-98df-0ca7b0184701 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:40.268364 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ff298-8126-4b8e-b3da-57473079fd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:40.269122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ff298-8126-4b8e-b3da-57473079fd83 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:40.269473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ff298-8126-4b8e-b3da-57473079fd83 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:40.269819 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84ff298-8126-4b8e-b3da-57473079fd83 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:40.270722 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ff298-8126-4b8e-b3da-57473079fd83 None admin] RBAC: Adding query filter params (role.id=83bf7d9c29d4464dab8030f6d706cd6d, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=f81aecdc87d84501ba94b208405b51dd) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:40.271113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ff298-8126-4b8e-b3da-57473079fd83 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:40.272226 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867fb99-7ade-447f-9f7f-694f9d0de5b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:40.272703 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84ff298-8126-4b8e-b3da-57473079fd83 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:40.273166 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867fb99-7ade-447f-9f7f-694f9d0de5b1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:40.273522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867fb99-7ade-447f-9f7f-694f9d0de5b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:40.273870 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9867fb99-7ade-447f-9f7f-694f9d0de5b1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:40.284216 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 336/681] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:54:40 2022] GET /identity/v3/role_assignments?user.id=f81aecdc87d84501ba94b208405b51dd&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=83bf7d9c29d4464dab8030f6d706cd6d => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:40.284216 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ff298-8126-4b8e-b3da-57473079fd83 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:40.290341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9867fb99-7ade-447f-9f7f-694f9d0de5b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:40.298589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9867fb99-7ade-447f-9f7f-694f9d0de5b1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:40.298589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:40.298589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:40.298589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:40.298589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:40.298589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:40.298589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:40.298589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:40.298589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:40.298589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:40.298589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:40.298589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:40.298589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:40.298589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:40.298589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:40.298589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:40.299602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:40.299602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:40.299602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:40.299602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:40.299602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:40.299602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:40.299602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:40.299602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:40.299602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:40.299602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:40.299602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:40.299602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:40.299602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:40.299602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:40.299602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:40.299602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:40.300555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:40.300555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:40.300555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:40.300555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:40.300555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:40.300555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:40.300555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:40.300555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:40.300555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:40.300555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:40.300555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:40.300555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:40.300555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 346/682] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:54:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:40.301259 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867fb99-7ade-447f-9f7f-694f9d0de5b1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:40.422642 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84ff298-8126-4b8e-b3da-57473079fd83 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:40.455155 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9867fb99-7ade-447f-9f7f-694f9d0de5b1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:40.461061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29797b-e65d-4b93-a5e6-b21ad5e36544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:40.461886 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29797b-e65d-4b93-a5e6-b21ad5e36544 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:40.462286 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29797b-e65d-4b93-a5e6-b21ad5e36544 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:40.462686 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af29797b-e65d-4b93-a5e6-b21ad5e36544 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:40.463532 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29797b-e65d-4b93-a5e6-b21ad5e36544 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:40.464000 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29797b-e65d-4b93-a5e6-b21ad5e36544 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:40.465914 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af29797b-e65d-4b93-a5e6-b21ad5e36544 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:40.485483 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 337/683] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:40.485483 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29797b-e65d-4b93-a5e6-b21ad5e36544 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:40.503804 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df315a34-73ed-4841-bd90-178950df1b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:40.504558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df315a34-73ed-4841-bd90-178950df1b18 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:40.504847 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df315a34-73ed-4841-bd90-178950df1b18 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:40.505130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df315a34-73ed-4841-bd90-178950df1b18 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:40.505937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df315a34-73ed-4841-bd90-178950df1b18 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:40.506251 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df315a34-73ed-4841-bd90-178950df1b18 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:40.507725 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df315a34-73ed-4841-bd90-178950df1b18 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:40.511341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:40.513061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:40.523839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 347/684] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:40.524692 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df315a34-73ed-4841-bd90-178950df1b18 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:40.620908 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af29797b-e65d-4b93-a5e6-b21ad5e36544 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:40.636595 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df315a34-73ed-4841-bd90-178950df1b18 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:40.659735 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a57e6-1be2-4613-80f8-9e272b9b85f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:40.660508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a57e6-1be2-4613-80f8-9e272b9b85f9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:40.660800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a57e6-1be2-4613-80f8-9e272b9b85f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:40.661079 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604a57e6-1be2-4613-80f8-9e272b9b85f9 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:40.661842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a57e6-1be2-4613-80f8-9e272b9b85f9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:40.662152 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a57e6-1be2-4613-80f8-9e272b9b85f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:40.663537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604a57e6-1be2-4613-80f8-9e272b9b85f9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:40.676008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 338/685] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:40.676790 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a57e6-1be2-4613-80f8-9e272b9b85f9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:40.683579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260d7ca-653e-4ba0-ba11-0184b45dc9f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:40.684765 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260d7ca-653e-4ba0-ba11-0184b45dc9f4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:40.685260 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260d7ca-653e-4ba0-ba11-0184b45dc9f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:40.685734 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8260d7ca-653e-4ba0-ba11-0184b45dc9f4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:40.696777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8260d7ca-653e-4ba0-ba11-0184b45dc9f4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:40.698869 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8260d7ca-653e-4ba0-ba11-0184b45dc9f4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:40.719743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8260d7ca-653e-4ba0-ba11-0184b45dc9f4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 15:54:40.719743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:40.719743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:40.719743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:40.719743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:40.719743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:40.719743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:40.719743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:40.719743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:40.719743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:40.719743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:40.719743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:40.719743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:40.719743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:40.719743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:40.719743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:40.720761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:40.721705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:40.721705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:40.721705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:40.721705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:40.721705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:40.721705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:40.721705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:40.721705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:40.721705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 15:54:40.721705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:40.724761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 348/686] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:54:40 2022] GET /identity/v3/users/swift => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:40.726038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260d7ca-653e-4ba0-ba11-0184b45dc9f4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:40.782662 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:40.786922 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:40.807013 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604a57e6-1be2-4613-80f8-9e272b9b85f9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:40.837926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8260d7ca-653e-4ba0-ba11-0184b45dc9f4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:40.849279 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534fa15-b5e2-440c-be19-9da5ffd37ed4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:40.850586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534fa15-b5e2-440c-be19-9da5ffd37ed4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:40.851273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534fa15-b5e2-440c-be19-9da5ffd37ed4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:40.851686 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f534fa15-b5e2-440c-be19-9da5ffd37ed4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:40.852705 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534fa15-b5e2-440c-be19-9da5ffd37ed4 None admin] RBAC: Adding query filter params (role.id=83bf7d9c29d4464dab8030f6d706cd6d, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=413b3d65427b41dc99bc7b0a94fb1dc8) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:40.853385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534fa15-b5e2-440c-be19-9da5ffd37ed4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:40.855729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f534fa15-b5e2-440c-be19-9da5ffd37ed4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:40.872449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 339/687] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:54:40 2022] GET /identity/v3/role_assignments?user.id=413b3d65427b41dc99bc7b0a94fb1dc8&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=83bf7d9c29d4464dab8030f6d706cd6d => generated 608 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:40.872449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534fa15-b5e2-440c-be19-9da5ffd37ed4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:40.894533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0444b-be6d-4853-8710-30cf80c9ea82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:40.895573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0444b-be6d-4853-8710-30cf80c9ea82 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:40.895985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0444b-be6d-4853-8710-30cf80c9ea82 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:40.896388 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0444b-be6d-4853-8710-30cf80c9ea82 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:40.917730 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0444b-be6d-4853-8710-30cf80c9ea82 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:40.919651 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0444b-be6d-4853-8710-30cf80c9ea82 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:40.950295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-12d0444b-be6d-4853-8710-30cf80c9ea82 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:40.950295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:40.950295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:40.950295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:40.950295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:40.950295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:40.950295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:40.950295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:40.950295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:40.950295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:40.950295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:40.950295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:40.950295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:40.950295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:40.950295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:40.951406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:40.951406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:40.951406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:40.951406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:40.951406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:40.951406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:40.951406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:40.951406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:40.951406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:40.951406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:40.951406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:40.951406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:40.951406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:40.951406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:40.951406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:40.951406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:40.951406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:40.952531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:40.952531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:40.952531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:40.952531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:40.954102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 349/688] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:40.960114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0444b-be6d-4853-8710-30cf80c9ea82 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:41.083113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0444b-be6d-4853-8710-30cf80c9ea82 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:41.095697 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:41.104940 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f534fa15-b5e2-440c-be19-9da5ffd37ed4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:41.127035 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89640a10-110c-49fe-905e-69e81a4424cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:41.127712 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89640a10-110c-49fe-905e-69e81a4424cb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:41.128031 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89640a10-110c-49fe-905e-69e81a4424cb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:41.128313 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89640a10-110c-49fe-905e-69e81a4424cb None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:41.129046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89640a10-110c-49fe-905e-69e81a4424cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:41.129360 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89640a10-110c-49fe-905e-69e81a4424cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:41.131214 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89640a10-110c-49fe-905e-69e81a4424cb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:41.145391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabf483-85d0-4b43-8280-16a2938cf127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:41.146154 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 350/689] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:41.146645 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabf483-85d0-4b43-8280-16a2938cf127 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:41.147004 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabf483-85d0-4b43-8280-16a2938cf127 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:41.147355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aabf483-85d0-4b43-8280-16a2938cf127 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:41.152578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabf483-85d0-4b43-8280-16a2938cf127 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:41.152783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabf483-85d0-4b43-8280-16a2938cf127 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:41.154804 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aabf483-85d0-4b43-8280-16a2938cf127 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:41.155982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89640a10-110c-49fe-905e-69e81a4424cb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:41.179018 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 340/690] 10.222.0.85 () {56 vars in 1135 bytes} [Wed May 25 15:54:40 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:41.185529 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabf483-85d0-4b43-8280-16a2938cf127 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:41.194883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:41.281662 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89640a10-110c-49fe-905e-69e81a4424cb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:41.313283 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aabf483-85d0-4b43-8280-16a2938cf127 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:41.322146 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e519f2d-9591-41a8-b4d5-a0774104dfa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:41.323274 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e519f2d-9591-41a8-b4d5-a0774104dfa0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:41.323763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e519f2d-9591-41a8-b4d5-a0774104dfa0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:41.324276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e519f2d-9591-41a8-b4d5-a0774104dfa0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:41.333156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e519f2d-9591-41a8-b4d5-a0774104dfa0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:41.334831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e519f2d-9591-41a8-b4d5-a0774104dfa0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:41.350380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8e519f2d-9591-41a8-b4d5-a0774104dfa0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 15:54:41.350380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:41.350380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:41.350380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:41.350380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:41.350380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:41.350380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:41.350380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:41.350380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:41.350380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:41.350380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:41.350380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:41.350380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:41.350380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:41.350380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:41.350380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:41.351396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:41.352402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:41.352402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:41.352402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:41.352402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:41.352402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:41.352402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:41.352402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:41.352402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:41.352402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 15:54:41.352402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:41.355143 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 351/691] 10.222.0.85 () {56 vars in 1093 bytes} [Wed May 25 15:54:41 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:41.360893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e519f2d-9591-41a8-b4d5-a0774104dfa0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:41.365536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352892c9-11be-4ff1-8645-d20c36aa0c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:41.365773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352892c9-11be-4ff1-8645-d20c36aa0c2b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:41.366024 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352892c9-11be-4ff1-8645-d20c36aa0c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:41.366307 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352892c9-11be-4ff1-8645-d20c36aa0c2b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:41.398100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:41.473502 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e519f2d-9591-41a8-b4d5-a0774104dfa0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:41.510332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5662e0-a8ba-444b-a4f0-11834ae76c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:41.511503 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5662e0-a8ba-444b-a4f0-11834ae76c85 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:41.512019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5662e0-a8ba-444b-a4f0-11834ae76c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:41.512584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5662e0-a8ba-444b-a4f0-11834ae76c85 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:41.513637 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5662e0-a8ba-444b-a4f0-11834ae76c85 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:41.514148 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5662e0-a8ba-444b-a4f0-11834ae76c85 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:41.516057 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5662e0-a8ba-444b-a4f0-11834ae76c85 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:41.530800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352892c9-11be-4ff1-8645-d20c36aa0c2b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:41.532384 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-352892c9-11be-4ff1-8645-d20c36aa0c2b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:41.537446 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 352/692] 10.222.0.85 () {56 vars in 1133 bytes} [Wed May 25 15:54:41 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:41.543036 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5662e0-a8ba-444b-a4f0-11834ae76c85 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:41.545511 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:41.640883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:41.674852 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5662e0-a8ba-444b-a4f0-11834ae76c85 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:41.692203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 341/693] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 507 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:54:41.697550 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352892c9-11be-4ff1-8645-d20c36aa0c2b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:41.741195 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2411d78-79f7-4d5e-922c-19141cf5fe23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:41.741861 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2411d78-79f7-4d5e-922c-19141cf5fe23 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:41.742151 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2411d78-79f7-4d5e-922c-19141cf5fe23 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:41.742431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2411d78-79f7-4d5e-922c-19141cf5fe23 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:41.825560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352892c9-11be-4ff1-8645-d20c36aa0c2b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:41.844434 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:41.892129 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2411d78-79f7-4d5e-922c-19141cf5fe23 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:41.893635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2411d78-79f7-4d5e-922c-19141cf5fe23 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:41.900518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3254f-668c-4b1f-a02b-02a8e9965ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:41.901346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3254f-668c-4b1f-a02b-02a8e9965ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:41.901632 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3254f-668c-4b1f-a02b-02a8e9965ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:41.901909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c3254f-668c-4b1f-a02b-02a8e9965ba7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:41.925821 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3254f-668c-4b1f-a02b-02a8e9965ba7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:41.927303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c3254f-668c-4b1f-a02b-02a8e9965ba7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:41.965453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-b1c3254f-668c-4b1f-a02b-02a8e9965ba7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:41.965453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:41.965453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:41.965453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:41.965453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:41.965453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:41.965453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:41.965453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:41.965453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:41.965453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:41.965453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:41.965453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:41.965453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:41.965453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:41.965453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:41.966474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:41.966474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:41.966474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:41.966474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:41.966474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:41.966474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:41.966474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:41.966474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:41.966474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:41.966474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:41.966474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:41.966474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:41.966474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:41.966474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:41.966474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:41.966474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:41.966474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:41.967375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:41.967375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:41.967375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:41.967375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:41.976475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 342/694] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 279 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:54:41.982186 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3254f-668c-4b1f-a02b-02a8e9965ba7 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:42.142527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:42.170585 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 353/695] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 628 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) May 25 15:54:42.177234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2411d78-79f7-4d5e-922c-19141cf5fe23 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:42.184864 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c3254f-668c-4b1f-a02b-02a8e9965ba7 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:42.236402 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5dcb90-546b-4097-b034-2e83e69b7946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:42.237487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dcb90-546b-4097-b034-2e83e69b7946 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:42.237934 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dcb90-546b-4097-b034-2e83e69b7946 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:42.238355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5dcb90-546b-4097-b034-2e83e69b7946 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:42.239096 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dcb90-546b-4097-b034-2e83e69b7946 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:42.239687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dcb90-546b-4097-b034-2e83e69b7946 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:42.241858 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5dcb90-546b-4097-b034-2e83e69b7946 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:42.258052 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 343/696] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:42.259549 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e11570-4ddd-4a29-970e-6a890355e1da None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:42.259865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e11570-4ddd-4a29-970e-6a890355e1da None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:42.260158 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e11570-4ddd-4a29-970e-6a890355e1da None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:42.263965 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 344/697] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:42 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:42.264731 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e11570-4ddd-4a29-970e-6a890355e1da None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:42.286564 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:42.306888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2411d78-79f7-4d5e-922c-19141cf5fe23 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:42.345323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d35890-e758-4d04-a116-18a3a0bd3948 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:42.346427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d35890-e758-4d04-a116-18a3a0bd3948 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:42.346903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d35890-e758-4d04-a116-18a3a0bd3948 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:42.347322 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d35890-e758-4d04-a116-18a3a0bd3948 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:42.359477 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d35890-e758-4d04-a116-18a3a0bd3948 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:42.361466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5d35890-e758-4d04-a116-18a3a0bd3948 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:42.382113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-b5d35890-e758-4d04-a116-18a3a0bd3948 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:42.382113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:42.382113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:42.382113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:42.382113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:42.382113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:42.382113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:42.382113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:42.382113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:42.382113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:42.382113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:42.382113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:42.382113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:42.382113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:42.382113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:42.383289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:42.383289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:42.383289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:42.383289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:42.383289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:42.383289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:42.383289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:42.383289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:42.383289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:42.383289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:42.383289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:42.383289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:42.383289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:42.383289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:42.383289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:42.383289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:42.383289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:42.384359 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:42.384359 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:42.384359 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:42.384359 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:42.390929 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e11570-4ddd-4a29-970e-6a890355e1da None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:42.391141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 354/698] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:42.392812 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb15e12-e804-4c23-9a54-7bafaf0c2a3b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:42.393272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb15e12-e804-4c23-9a54-7bafaf0c2a3b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:42.393803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eb15e12-e804-4c23-9a54-7bafaf0c2a3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:42.413511 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8eb15e12-e804-4c23-9a54-7bafaf0c2a3b None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:42.415049 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8eb15e12-e804-4c23-9a54-7bafaf0c2a3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:42.438561 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd9c66-b83f-4fb9-bfb8-18b370d1b335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:42.439401 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd9c66-b83f-4fb9-bfb8-18b370d1b335 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:42.439810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd9c66-b83f-4fb9-bfb8-18b370d1b335 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:42.440121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4edd9c66-b83f-4fb9-bfb8-18b370d1b335 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:42.448270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd9c66-b83f-4fb9-bfb8-18b370d1b335 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:42.450109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4edd9c66-b83f-4fb9-bfb8-18b370d1b335 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:42.452330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:42.466982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-4edd9c66-b83f-4fb9-bfb8-18b370d1b335 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:42.466982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:42.466982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:42.466982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:42.466982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:42.466982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:42.466982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:42.466982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:42.466982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:42.466982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:42.466982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:42.466982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:42.466982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:42.466982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:42.466982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:42.468156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:42.468156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:42.468156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:42.468156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:42.468156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:42.468156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:42.468156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:42.468156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:42.468156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:42.468156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:42.468156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:42.468156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:42.468156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:42.468156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:42.468156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:42.468156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:42.468156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:42.469324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:42.469324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:42.469324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:42.469324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:42.469324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:42.469324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:42.469324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:42.469324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:42.471508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 345/699] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:42.472640 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd9c66-b83f-4fb9-bfb8-18b370d1b335 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:42.499532 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 355/700] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:42.500716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb15e12-e804-4c23-9a54-7bafaf0c2a3b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:42.573962 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:42.599191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4edd9c66-b83f-4fb9-bfb8-18b370d1b335 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:42.623985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eb15e12-e804-4c23-9a54-7bafaf0c2a3b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:42.644790 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a7181-5233-43b9-98cc-01a5a02eaf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:42.645388 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a7181-5233-43b9-98cc-01a5a02eaf5f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:42.645800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a7181-5233-43b9-98cc-01a5a02eaf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:42.646092 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512a7181-5233-43b9-98cc-01a5a02eaf5f None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:42.646831 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a7181-5233-43b9-98cc-01a5a02eaf5f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:42.647151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a7181-5233-43b9-98cc-01a5a02eaf5f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:42.649015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512a7181-5233-43b9-98cc-01a5a02eaf5f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:42.663049 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a588b4-4b16-4259-8497-4f7e30c7586d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:42.664124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a588b4-4b16-4259-8497-4f7e30c7586d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:42.664666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a588b4-4b16-4259-8497-4f7e30c7586d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:42.665162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49a588b4-4b16-4259-8497-4f7e30c7586d None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:42.666153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a588b4-4b16-4259-8497-4f7e30c7586d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:42.666662 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a588b4-4b16-4259-8497-4f7e30c7586d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:42.668398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49a588b4-4b16-4259-8497-4f7e30c7586d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:42.668920 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 346/701] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:42.669719 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a7181-5233-43b9-98cc-01a5a02eaf5f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:42.682467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 356/702] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:42.683791 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a588b4-4b16-4259-8497-4f7e30c7586d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:42.716350 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:42.791875 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512a7181-5233-43b9-98cc-01a5a02eaf5f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:42.800324 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49a588b4-4b16-4259-8497-4f7e30c7586d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:42.828425 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce15541-3a3d-408e-932b-abd71c46f7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:42.829231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce15541-3a3d-408e-932b-abd71c46f7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:42.829660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce15541-3a3d-408e-932b-abd71c46f7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:42.829792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce15541-3a3d-408e-932b-abd71c46f7e0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:42.841005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:42.843375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0d5175-ba39-40ac-811b-c3bf3795cdd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:42.843757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce15541-3a3d-408e-932b-abd71c46f7e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:42.844486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d5175-ba39-40ac-811b-c3bf3795cdd0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:42.844750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d5175-ba39-40ac-811b-c3bf3795cdd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:42.845029 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0d5175-ba39-40ac-811b-c3bf3795cdd0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:42.845797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce15541-3a3d-408e-932b-abd71c46f7e0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:42.853162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d5175-ba39-40ac-811b-c3bf3795cdd0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:42.855038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0d5175-ba39-40ac-811b-c3bf3795cdd0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:42.864742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-0ce15541-3a3d-408e-932b-abd71c46f7e0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:42.864742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:42.864742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:42.864742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:42.864742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:42.864742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:42.864742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:42.864742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:42.864742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:42.864742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:42.864742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:42.864742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:42.864742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:42.864742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:42.864742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:42.865701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:42.865701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:42.865701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:42.865701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:42.865701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:42.865701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:42.865701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:42.865701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:42.865701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:42.865701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:42.865701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:42.865701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:42.865701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:42.865701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:42.865701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:42.865701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:42.865701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:42.871592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:42.871592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:42.871592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:42.871592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:42.871592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 347/703] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:42.871592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce15541-3a3d-408e-932b-abd71c46f7e0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:42.872055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-6b0d5175-ba39-40ac-811b-c3bf3795cdd0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:42.872055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:42.872055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:42.872055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:42.872055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:42.872055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:42.872055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:42.872055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:42.872055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:42.872055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:42.872055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:42.872055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:42.872055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:42.872055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:42.872055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:42.872853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:42.872853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:42.872853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:42.872853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:42.872853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:42.872853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:42.872853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:42.872853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:42.872853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:42.872853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:42.872853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:42.872853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:42.872853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:42.872853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:42.872853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:42.872853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:42.872853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:42.873733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:42.873733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:42.873733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:42.873733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:42.873733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:42.873733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:42.873733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:42.873733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:42.876960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 357/704] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:42.878312 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb61d0-e6db-4630-a9ed-5c13a9b8e80e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:42.878608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb61d0-e6db-4630-a9ed-5c13a9b8e80e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:42.878891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eb61d0-e6db-4630-a9ed-5c13a9b8e80e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:42.880335 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 358/705] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:42.880890 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb61d0-e6db-4630-a9ed-5c13a9b8e80e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:42.973746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:42.985649 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce15541-3a3d-408e-932b-abd71c46f7e0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:42.987023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eb61d0-e6db-4630-a9ed-5c13a9b8e80e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:43.022231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987f7ab-5bd0-47e1-8dac-482bf8ea9668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:43.022892 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987f7ab-5bd0-47e1-8dac-482bf8ea9668 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:43.023183 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987f7ab-5bd0-47e1-8dac-482bf8ea9668 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:43.023466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b987f7ab-5bd0-47e1-8dac-482bf8ea9668 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:43.024343 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b987f7ab-5bd0-47e1-8dac-482bf8ea9668 None admin] RBAC: Adding query filter params (role.id=83bf7d9c29d4464dab8030f6d706cd6d, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=77fda27641c84fccb68a5cdaf4e8dc4b) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:43.024578 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423e0420-27f6-4ad3-9a5c-93fc2b5fea5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:43.024729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b987f7ab-5bd0-47e1-8dac-482bf8ea9668 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:43.025245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e0420-27f6-4ad3-9a5c-93fc2b5fea5b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:43.025531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e0420-27f6-4ad3-9a5c-93fc2b5fea5b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:43.025802 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423e0420-27f6-4ad3-9a5c-93fc2b5fea5b None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:43.026101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b987f7ab-5bd0-47e1-8dac-482bf8ea9668 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:43.026527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e0420-27f6-4ad3-9a5c-93fc2b5fea5b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:43.026846 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e0420-27f6-4ad3-9a5c-93fc2b5fea5b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:43.028277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423e0420-27f6-4ad3-9a5c-93fc2b5fea5b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:43.044033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 348/706] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:54:42 2022] GET /identity/v3/role_assignments?user.id=77fda27641c84fccb68a5cdaf4e8dc4b&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=83bf7d9c29d4464dab8030f6d706cd6d => generated 608 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:43.044033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987f7ab-5bd0-47e1-8dac-482bf8ea9668 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:43.050890 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 359/707] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:43.052346 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848ba47-799b-498c-9dae-bc7bf6dc5178 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:43.052635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848ba47-799b-498c-9dae-bc7bf6dc5178 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:43.052922 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1848ba47-799b-498c-9dae-bc7bf6dc5178 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:43.074783 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1848ba47-799b-498c-9dae-bc7bf6dc5178 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:43.076085 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1848ba47-799b-498c-9dae-bc7bf6dc5178 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:43.133502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:43.157457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 360/708] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:43.158337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848ba47-799b-498c-9dae-bc7bf6dc5178 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:43.213320 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b987f7ab-5bd0-47e1-8dac-482bf8ea9668 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:43.254930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:43.262642 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be595b4-af1d-4667-8311-46803d941d80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:43.264061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be595b4-af1d-4667-8311-46803d941d80 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:43.264061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be595b4-af1d-4667-8311-46803d941d80 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:43.264061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be595b4-af1d-4667-8311-46803d941d80 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:43.265187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be595b4-af1d-4667-8311-46803d941d80 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:43.265516 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be595b4-af1d-4667-8311-46803d941d80 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:43.267017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be595b4-af1d-4667-8311-46803d941d80 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:43.282321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1848ba47-799b-498c-9dae-bc7bf6dc5178 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:43.283080 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 349/709] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:43.283782 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be595b4-af1d-4667-8311-46803d941d80 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:43.326641 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454bfed-b596-4fae-9e49-a814fce747a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:43.327000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454bfed-b596-4fae-9e49-a814fce747a4 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:43.328032 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454bfed-b596-4fae-9e49-a814fce747a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:43.328032 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6454bfed-b596-4fae-9e49-a814fce747a4 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:43.328551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454bfed-b596-4fae-9e49-a814fce747a4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:43.330195 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6454bfed-b596-4fae-9e49-a814fce747a4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:43.359057 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 361/710] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:54:43 2022] POST /identity/v3/users => generated 254 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:54:43.359621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454bfed-b596-4fae-9e49-a814fce747a4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:43.417411 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be595b4-af1d-4667-8311-46803d941d80 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:43.510712 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6454bfed-b596-4fae-9e49-a814fce747a4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:43.552321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b5b96d-ae8d-486c-8e19-94df2929e701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:43.553021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5b96d-ae8d-486c-8e19-94df2929e701 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:43.553319 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5b96d-ae8d-486c-8e19-94df2929e701 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:43.553611 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b5b96d-ae8d-486c-8e19-94df2929e701 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:43.554483 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5b96d-ae8d-486c-8e19-94df2929e701 None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=4751abc9f67947b3ad17ba2b78ce2c12) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:43.554802 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5b96d-ae8d-486c-8e19-94df2929e701 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:43.557151 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b5b96d-ae8d-486c-8e19-94df2929e701 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:43.560301 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:43.564097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:43.569411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 362/711] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:54:43 2022] GET /identity/v3/role_assignments?user.id=4751abc9f67947b3ad17ba2b78ce2c12&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 259 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:43.571087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a740f-3037-4635-96ba-6dba431b9af8 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:43.571578 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a740f-3037-4635-96ba-6dba431b9af8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:43.572077 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448a740f-3037-4635-96ba-6dba431b9af8 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:43.575465 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 363/712] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:43.583853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75664be3-4a0a-4f93-846d-5d54e69c992c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:43.583853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75664be3-4a0a-4f93-846d-5d54e69c992c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:43.583853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75664be3-4a0a-4f93-846d-5d54e69c992c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:43.603110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-75664be3-4a0a-4f93-846d-5d54e69c992c None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:43.604489 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-75664be3-4a0a-4f93-846d-5d54e69c992c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:43.619990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c243d3-c15b-44c3-a2e2-bd8ea1696094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:43.620847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c243d3-c15b-44c3-a2e2-bd8ea1696094 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:43.621129 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c243d3-c15b-44c3-a2e2-bd8ea1696094 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:43.621405 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4c243d3-c15b-44c3-a2e2-bd8ea1696094 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:43.622129 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c243d3-c15b-44c3-a2e2-bd8ea1696094 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:43.623752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4c243d3-c15b-44c3-a2e2-bd8ea1696094 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:43.651752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-e4c243d3-c15b-44c3-a2e2-bd8ea1696094 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 15:54:43.651752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:43.651752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:54:43.651752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:54:43.651752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:54:43.651752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:54:43.651752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:54:43.651752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:54:43.651752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 15:54:43.651752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:43.651752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:54:43.651752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:43.651752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:43.651752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:43.651752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:43.651752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:43.652944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:43.652944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:43.652944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:43.652944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:43.652944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:43.652944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:43.652944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:43.652944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:54:43.652944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:54:43.652944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:54:43.652944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:54:43.652944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:43.652944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:43.652944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:43.652944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:43.652944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:43.653801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:43.653801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:43.653801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:43.653801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:43.653801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:43.653801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:43.653801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:43.653801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:43.653801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:43.653801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:43.653801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:43.653801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:54:43.653801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:54:43.653801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 15:54:43.653801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:43.654963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 350/713] 10.222.0.85 () {56 vars in 1129 bytes} [Wed May 25 15:54:43 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 371 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:43.660162 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c243d3-c15b-44c3-a2e2-bd8ea1696094 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:43.692918 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 364/714] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:43.723592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75664be3-4a0a-4f93-846d-5d54e69c992c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:43.784798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4c243d3-c15b-44c3-a2e2-bd8ea1696094 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:43.824752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30870251-fa49-457d-917a-3a3721c74599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:43.825049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30870251-fa49-457d-917a-3a3721c74599 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:43.825315 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30870251-fa49-457d-917a-3a3721c74599 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:43.825591 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30870251-fa49-457d-917a-3a3721c74599 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:43.826426 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30870251-fa49-457d-917a-3a3721c74599 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:43.826739 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30870251-fa49-457d-917a-3a3721c74599 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:43.828169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30870251-fa49-457d-917a-3a3721c74599 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:43.838980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 351/715] 10.222.0.85 () {56 vars in 1124 bytes} [Wed May 25 15:54:43 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:43.840017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30870251-fa49-457d-917a-3a3721c74599 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:43.852357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75664be3-4a0a-4f93-846d-5d54e69c992c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:43.881760 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:43.892014 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed5298-10bb-4495-98e0-3d397320bd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:43.893271 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed5298-10bb-4495-98e0-3d397320bd00 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:43.893738 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed5298-10bb-4495-98e0-3d397320bd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:43.894215 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aed5298-10bb-4495-98e0-3d397320bd00 None admin] PATH_INFO: `/v3/services/network` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:43.895071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed5298-10bb-4495-98e0-3d397320bd00 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:43.896510 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:43.896851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aed5298-10bb-4495-98e0-3d397320bd00 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:43.925977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2aed5298-10bb-4495-98e0-3d397320bd00 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 15:54:43.925977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:43.925977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:54:43.925977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:54:43.925977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:54:43.925977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:54:43.925977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:54:43.925977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:54:43.925977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 15:54:43.925977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:43.925977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:54:43.925977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:43.925977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:43.925977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:43.925977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:43.925977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:43.925977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:43.927099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:43.927099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:43.927099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:43.927099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:43.927099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:43.927099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:43.927099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:54:43.927099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:54:43.927099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:54:43.927099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:54:43.927099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:43.927099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:43.927099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:43.927099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:43.927099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:43.927099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:43.928013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:43.928013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:43.928013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:43.928013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:43.928013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:43.928013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:43.928013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:43.928013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:43.928013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:43.928013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:43.928013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:54:43.928013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:54:43.928013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 15:54:43.928013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:43.931320 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 365/716] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:43 2022] GET /identity/v3/services/network => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:43.940503 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed5298-10bb-4495-98e0-3d397320bd00 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:43.994356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30870251-fa49-457d-917a-3a3721c74599 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:44.033806 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e9fe2-c0cd-403a-b9a9-05c0a6ca7902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:44.034875 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e9fe2-c0cd-403a-b9a9-05c0a6ca7902 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:44.035315 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e9fe2-c0cd-403a-b9a9-05c0a6ca7902 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:44.035735 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5e9fe2-c0cd-403a-b9a9-05c0a6ca7902 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:44.037217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e9fe2-c0cd-403a-b9a9-05c0a6ca7902 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:44.037697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e9fe2-c0cd-403a-b9a9-05c0a6ca7902 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:44.039520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5e9fe2-c0cd-403a-b9a9-05c0a6ca7902 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:44.047074 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 352/717] 10.222.0.85 () {56 vars in 1124 bytes} [Wed May 25 15:54:43 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:44.127466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aed5298-10bb-4495-98e0-3d397320bd00 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:44.190734 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adea32c2-a8f1-4d56-a2a4-31fe282d50e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:44.191423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea32c2-a8f1-4d56-a2a4-31fe282d50e0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:44.191716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea32c2-a8f1-4d56-a2a4-31fe282d50e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:44.192026 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adea32c2-a8f1-4d56-a2a4-31fe282d50e0 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:44.192782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea32c2-a8f1-4d56-a2a4-31fe282d50e0 None admin] RBAC: Adding query filter params (name=network) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:44.193100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea32c2-a8f1-4d56-a2a4-31fe282d50e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:44.194528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adea32c2-a8f1-4d56-a2a4-31fe282d50e0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:44.203954 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 366/718] 10.222.0.85 () {56 vars in 1112 bytes} [Wed May 25 15:54:43 2022] GET /identity/v3/services?name=network => generated 124 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:44.219796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e9fe2-c0cd-403a-b9a9-05c0a6ca7902 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:44.379675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5e9fe2-c0cd-403a-b9a9-05c0a6ca7902 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:44.387022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:44.414935 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3820d0c-ac53-4b7d-8e23-c9d1e61312f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:44.415861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3820d0c-ac53-4b7d-8e23-c9d1e61312f8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:44.416287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3820d0c-ac53-4b7d-8e23-c9d1e61312f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:44.416703 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3820d0c-ac53-4b7d-8e23-c9d1e61312f8 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:44.417553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3820d0c-ac53-4b7d-8e23-c9d1e61312f8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:44.418027 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3820d0c-ac53-4b7d-8e23-c9d1e61312f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:44.419578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3820d0c-ac53-4b7d-8e23-c9d1e61312f8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:44.424958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 353/719] 10.222.0.85 () {56 vars in 1112 bytes} [Wed May 25 15:54:44 2022] GET /identity/v3/services?type=network => generated 124 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:45.865604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476397b5-81c6-4595-87b5-e36900e3908d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:45.866172 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476397b5-81c6-4595-87b5-e36900e3908d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:45.866172 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476397b5-81c6-4595-87b5-e36900e3908d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:45.867771 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 367/720] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:45.873621 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606cab0-6120-4a68-8416-ea119bdac227 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:45.874035 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606cab0-6120-4a68-8416-ea119bdac227 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:45.874209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d606cab0-6120-4a68-8416-ea119bdac227 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:45.892331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d606cab0-6120-4a68-8416-ea119bdac227 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:45.893594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d606cab0-6120-4a68-8416-ea119bdac227 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:45.967771 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 354/721] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:45.991075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476397b5-81c6-4595-87b5-e36900e3908d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:46.071384 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:46.120274 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476397b5-81c6-4595-87b5-e36900e3908d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:46.154886 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13d04a20-c3eb-4f59-a531-f42ab0556de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:46.155703 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d04a20-c3eb-4f59-a531-f42ab0556de0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:46.156014 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d04a20-c3eb-4f59-a531-f42ab0556de0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:46.156297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13d04a20-c3eb-4f59-a531-f42ab0556de0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:46.170801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13d04a20-c3eb-4f59-a531-f42ab0556de0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:46.182768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-13d04a20-c3eb-4f59-a531-f42ab0556de0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:46.182768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:46.182768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:46.182768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:46.182768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:46.182768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:46.182768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:46.182768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:46.182768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:46.182768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:46.182768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:46.182768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:46.182768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:46.182768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:46.182768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:46.182768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:46.183786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:46.183786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:46.183786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:46.183786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:46.183786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:46.183786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:46.183786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:46.183786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:46.183786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:46.183786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:46.183786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:46.183786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:46.183786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:46.183786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:46.183786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:46.183786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:46.184908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:46.184908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:46.184908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:46.184908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:46.184908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:46.184908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:46.184908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:46.184908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:46.184908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:46.184908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:46.184908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:46.184908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:46.184908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 368/722] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:54:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:46.189935 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606cab0-6120-4a68-8416-ea119bdac227 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:46.231227 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da9aa7-5cd3-41c3-bbfc-5b694aa5bc7a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:46.231919 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da9aa7-5cd3-41c3-bbfc-5b694aa5bc7a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:46.232431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12da9aa7-5cd3-41c3-bbfc-5b694aa5bc7a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:46.235346 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 369/723] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:54:46.239621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4ecf5-46c2-4588-a5c2-6832f911a026 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:46.240192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4ecf5-46c2-4588-a5c2-6832f911a026 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:46.240682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbb4ecf5-46c2-4588-a5c2-6832f911a026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:46.273319 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bbb4ecf5-46c2-4588-a5c2-6832f911a026 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:46.274907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bbb4ecf5-46c2-4588-a5c2-6832f911a026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:46.294471 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:46.298091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d606cab0-6120-4a68-8416-ea119bdac227 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:46.335847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67866a43-09c8-4097-96b3-6b6653509956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:46.335847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67866a43-09c8-4097-96b3-6b6653509956 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:46.335847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67866a43-09c8-4097-96b3-6b6653509956 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:46.335847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67866a43-09c8-4097-96b3-6b6653509956 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:46.336259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67866a43-09c8-4097-96b3-6b6653509956 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:46.336259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67866a43-09c8-4097-96b3-6b6653509956 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:46.337676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67866a43-09c8-4097-96b3-6b6653509956 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:46.347505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 355/724] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:46.353148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b23e07-b254-459a-a503-bfb7c4602e8a None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:46.353443 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b23e07-b254-459a-a503-bfb7c4602e8a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:46.353728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b23e07-b254-459a-a503-bfb7c4602e8a None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:46.356260 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 356/725] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:46.356260 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b23e07-b254-459a-a503-bfb7c4602e8a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:46.404017 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 370/726] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) May 25 15:54:46.405851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cf1d6-1545-45b7-b276-6771dc01b3a6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:46.406328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cf1d6-1545-45b7-b276-6771dc01b3a6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:46.406794 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c55cf1d6-1545-45b7-b276-6771dc01b3a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:46.439355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c55cf1d6-1545-45b7-b276-6771dc01b3a6 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:46.443496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c55cf1d6-1545-45b7-b276-6771dc01b3a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:46.465257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b23e07-b254-459a-a503-bfb7c4602e8a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:46.493380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:46.507924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7d041-034d-43f4-97a2-dafac5edd2e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:46.508835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7d041-034d-43f4-97a2-dafac5edd2e6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:46.509192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7d041-034d-43f4-97a2-dafac5edd2e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:46.509530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3b7d041-034d-43f4-97a2-dafac5edd2e6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:46.524039 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7d041-034d-43f4-97a2-dafac5edd2e6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:46.525600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3b7d041-034d-43f4-97a2-dafac5edd2e6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:46.527225 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 371/727] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:46.528811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55cf1d6-1545-45b7-b276-6771dc01b3a6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:46.560449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-e3b7d041-034d-43f4-97a2-dafac5edd2e6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:46.560449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:46.560449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:46.560449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:46.560449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:46.560449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:46.560449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:46.560449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:46.560449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:46.560449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:46.560449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:46.560449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:46.560449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:46.560449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:46.560449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:46.561452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:46.561452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:46.561452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:46.561452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:46.561452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:46.561452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:46.561452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:46.561452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:46.561452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:46.561452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:46.561452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:46.561452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:46.561452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:46.561452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:46.561452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:46.561452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:46.561452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:46.562812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:46.562812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:46.562812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:46.562812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:46.562812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 357/728] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:46.562812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7d041-034d-43f4-97a2-dafac5edd2e6 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:46.585947 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:46.681428 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c55cf1d6-1545-45b7-b276-6771dc01b3a6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:46.691544 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3b7d041-034d-43f4-97a2-dafac5edd2e6 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:46.724086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d935f-24aa-4f55-896e-50f065a79ba7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:46.725865 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d935f-24aa-4f55-896e-50f065a79ba7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:46.726432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d935f-24aa-4f55-896e-50f065a79ba7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:46.726892 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769d935f-24aa-4f55-896e-50f065a79ba7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:46.729850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289135e2-ae95-4f36-879b-20502c1aac62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:46.730817 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289135e2-ae95-4f36-879b-20502c1aac62 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:46.731202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289135e2-ae95-4f36-879b-20502c1aac62 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:46.731578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289135e2-ae95-4f36-879b-20502c1aac62 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:46.739528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769d935f-24aa-4f55-896e-50f065a79ba7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:46.742982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289135e2-ae95-4f36-879b-20502c1aac62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:46.748496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-769d935f-24aa-4f55-896e-50f065a79ba7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:46.748496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:46.748496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:46.748496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:46.748496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:46.748496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:46.748496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:46.748496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:46.748496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:46.748496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:46.748496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:46.748496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:46.748496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:46.748496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:46.748496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:46.748496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:46.749660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:46.749660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:46.749660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:46.749660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:46.749660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:46.749660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:46.749660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:46.749660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:46.749660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:46.749660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:46.749660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:46.749660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:46.749660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:46.749660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:46.749660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:46.749660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:46.750723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:46.750723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:46.750723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:46.750723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:46.750723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:46.750723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:46.750723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:46.750723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:46.750723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:46.750723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:46.750723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:46.750723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:46.753893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-289135e2-ae95-4f36-879b-20502c1aac62 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:46.753893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:46.753893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:46.753893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:46.753893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:46.753893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:46.753893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:46.753893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:46.753893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:46.753893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:46.753893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:46.753893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:46.753893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:46.753893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:46.753893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:46.753893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:46.755031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:46.755031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:46.755031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:46.755031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:46.755031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:46.755031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:46.755031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:46.755031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:46.755031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:46.755031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:46.755031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:46.755031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:46.755031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:46.755031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:46.755031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:46.755031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:46.756124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:46.756124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:46.756124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:46.756124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:46.756124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:46.756124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:46.756124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:46.756124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:46.756124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:46.756124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:46.756124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:46.756124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:46.760478 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 358/729] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:54:46 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:46.760478 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289135e2-ae95-4f36-879b-20502c1aac62 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:46.764481 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 372/730] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:46 2022] GET /identity/v3/roles/service => generated 85 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:46.766750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d935f-24aa-4f55-896e-50f065a79ba7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:46.824151 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:46.863894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:46.895276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769d935f-24aa-4f55-896e-50f065a79ba7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:46.900143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289135e2-ae95-4f36-879b-20502c1aac62 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:46.938518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a69c7-a8e6-41cc-9bbe-9853f129ce49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:46.939116 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a69c7-a8e6-41cc-9bbe-9853f129ce49 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:46.939407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a69c7-a8e6-41cc-9bbe-9853f129ce49 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:46.939690 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791a69c7-a8e6-41cc-9bbe-9853f129ce49 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:46.940453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a69c7-a8e6-41cc-9bbe-9853f129ce49 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:46.940768 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a69c7-a8e6-41cc-9bbe-9853f129ce49 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:46.942432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791a69c7-a8e6-41cc-9bbe-9853f129ce49 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:46.945584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af857dc0-9d3a-4598-995d-e57c0282e628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:46.946521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af857dc0-9d3a-4598-995d-e57c0282e628 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:46.947019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af857dc0-9d3a-4598-995d-e57c0282e628 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:46.947522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af857dc0-9d3a-4598-995d-e57c0282e628 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:46.948539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af857dc0-9d3a-4598-995d-e57c0282e628 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:46.949120 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af857dc0-9d3a-4598-995d-e57c0282e628 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:46.951550 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af857dc0-9d3a-4598-995d-e57c0282e628 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:46.956494 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 359/731] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:46.957712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a69c7-a8e6-41cc-9bbe-9853f129ce49 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:46.968560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 373/732] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:46.969013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af857dc0-9d3a-4598-995d-e57c0282e628 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:47.075199 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:47.076748 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791a69c7-a8e6-41cc-9bbe-9853f129ce49 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:47.079327 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af857dc0-9d3a-4598-995d-e57c0282e628 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:47.111500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f77cf8-3429-46b2-a488-74fb169c10e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:47.112226 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f77cf8-3429-46b2-a488-74fb169c10e4 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:47.112519 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f77cf8-3429-46b2-a488-74fb169c10e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:47.113037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f77cf8-3429-46b2-a488-74fb169c10e4 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:47.113711 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f77cf8-3429-46b2-a488-74fb169c10e4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:47.114028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f77cf8-3429-46b2-a488-74fb169c10e4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:47.115407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f77cf8-3429-46b2-a488-74fb169c10e4 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:47.116675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8fce7f-f63b-45b3-8dd5-e015fc7350a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:47.117016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:47.117480 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fce7f-f63b-45b3-8dd5-e015fc7350a3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:47.117767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fce7f-f63b-45b3-8dd5-e015fc7350a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:47.118049 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8fce7f-f63b-45b3-8dd5-e015fc7350a3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:47.125619 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 360/733] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:47.126657 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f77cf8-3429-46b2-a488-74fb169c10e4 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:47.131335 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8fce7f-f63b-45b3-8dd5-e015fc7350a3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:47.134441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8fce7f-f63b-45b3-8dd5-e015fc7350a3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:47.149339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0c8fce7f-f63b-45b3-8dd5-e015fc7350a3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 15:54:47.149339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:47.149339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:47.149339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:47.149339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:47.149339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:47.149339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:47.149339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:47.149339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:47.149339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:47.149339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:47.149339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:47.149339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:47.149339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:47.149339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:47.149339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:47.150349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:47.151315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:47.151315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:47.151315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:47.151315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:47.151315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:47.151315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:47.151315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:47.151315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:47.151315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 15:54:47.151315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:47.151315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 374/734] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:54:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:47.151947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250575c-6d55-44ec-be25-44a5eb45baca None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:47.152068 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250575c-6d55-44ec-be25-44a5eb45baca None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:47.152353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2250575c-6d55-44ec-be25-44a5eb45baca None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:47.153747 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 375/735] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:47.154309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250575c-6d55-44ec-be25-44a5eb45baca None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:47.230258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f77cf8-3429-46b2-a488-74fb169c10e4 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:47.256390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2250575c-6d55-44ec-be25-44a5eb45baca None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:47.263058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d581448a-f25c-43ee-a092-3bc3574af093 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:47.263888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581448a-f25c-43ee-a092-3bc3574af093 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:47.264181 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581448a-f25c-43ee-a092-3bc3574af093 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:47.264466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d581448a-f25c-43ee-a092-3bc3574af093 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:47.275287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581448a-f25c-43ee-a092-3bc3574af093 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:47.276760 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d581448a-f25c-43ee-a092-3bc3574af093 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:47.293957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db5b7b7e-6b57-4c3e-8456-cffa6cfcb6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:47.294508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-d581448a-f25c-43ee-a092-3bc3574af093 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:47.294508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:47.294508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:47.294508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:47.294508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:47.294508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:47.294508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:47.294508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:47.294508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:47.294508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:47.294508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:47.294508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:47.294508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:47.294508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:47.294508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:47.295558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:47.295558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:47.295558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:47.295558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:47.295558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:47.295558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:47.295558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:47.295558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:47.295558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:47.295558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:47.295558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:47.295558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:47.295558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:47.295558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:47.295558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:47.295558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:47.295558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:47.296782 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:47.296782 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:47.296782 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:47.296782 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:47.297776 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b7b7e-6b57-4c3e-8456-cffa6cfcb6ea None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:47.298071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b7b7e-6b57-4c3e-8456-cffa6cfcb6ea None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:47.298355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db5b7b7e-6b57-4c3e-8456-cffa6cfcb6ea None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:47.300510 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 361/736] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:47.301969 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2a2f8-6049-406e-bd25-ee514958e1bb None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:47.302075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2a2f8-6049-406e-bd25-ee514958e1bb None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:47.302348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a2a2f8-6049-406e-bd25-ee514958e1bb None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:47.303670 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 362/737] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:47.304401 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2a2f8-6049-406e-bd25-ee514958e1bb None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:47.311677 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b7b7e-6b57-4c3e-8456-cffa6cfcb6ea None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:47.313226 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db5b7b7e-6b57-4c3e-8456-cffa6cfcb6ea None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:47.331499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-db5b7b7e-6b57-4c3e-8456-cffa6cfcb6ea None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:47.331499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:47.331499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:47.331499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:47.331499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:47.331499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:47.331499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:47.331499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:47.331499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:47.331499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:47.331499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:47.331499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:47.331499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:47.331499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:47.331499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:47.332797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:47.332797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:47.332797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:47.332797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:47.332797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:47.332797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:47.332797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:47.332797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:47.332797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:47.332797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:47.332797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:47.332797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:47.332797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:47.332797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:47.332797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:47.332797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:47.332797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:47.333700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:47.333700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:47.333700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:47.333700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:47.333700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 376/738] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:47.334200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74115974-a066-4c32-9b38-77eec9620f8a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:47.334531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74115974-a066-4c32-9b38-77eec9620f8a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:47.334816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74115974-a066-4c32-9b38-77eec9620f8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:47.352128 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-74115974-a066-4c32-9b38-77eec9620f8a None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:47.353361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-74115974-a066-4c32-9b38-77eec9620f8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:47.357035 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:47.361431 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:47.411565 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a2a2f8-6049-406e-bd25-ee514958e1bb None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:47.423255 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 377/739] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:47.424135 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74115974-a066-4c32-9b38-77eec9620f8a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:47.447494 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce77a09-ee58-4a3f-b5ef-8aa3cf356685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:47.448216 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce77a09-ee58-4a3f-b5ef-8aa3cf356685 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:47.448508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce77a09-ee58-4a3f-b5ef-8aa3cf356685 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:47.448791 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce77a09-ee58-4a3f-b5ef-8aa3cf356685 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:47.449677 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce77a09-ee58-4a3f-b5ef-8aa3cf356685 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:47.449997 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce77a09-ee58-4a3f-b5ef-8aa3cf356685 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:47.451454 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce77a09-ee58-4a3f-b5ef-8aa3cf356685 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:47.469030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 363/740] 10.222.0.85 () {56 vars in 1135 bytes} [Wed May 25 15:54:47 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:47.470276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f33d06-b9bd-4e53-b912-119cdebeecc4 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:47.470565 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f33d06-b9bd-4e53-b912-119cdebeecc4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:47.470842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f33d06-b9bd-4e53-b912-119cdebeecc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:47.486624 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-03f33d06-b9bd-4e53-b912-119cdebeecc4 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:47.487753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-03f33d06-b9bd-4e53-b912-119cdebeecc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:47.528067 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74115974-a066-4c32-9b38-77eec9620f8a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:47.556240 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 364/741] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) May 25 15:54:47.557072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f33d06-b9bd-4e53-b912-119cdebeecc4 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:47.562988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc0120-a3c2-4a29-b3aa-93f1bd339e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:47.563664 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc0120-a3c2-4a29-b3aa-93f1bd339e35 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:47.563978 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc0120-a3c2-4a29-b3aa-93f1bd339e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:47.564261 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cecc0120-a3c2-4a29-b3aa-93f1bd339e35 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:47.564988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc0120-a3c2-4a29-b3aa-93f1bd339e35 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:47.565286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc0120-a3c2-4a29-b3aa-93f1bd339e35 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:47.566673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cecc0120-a3c2-4a29-b3aa-93f1bd339e35 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:47.574926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:47.579210 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 378/742] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:47.580013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc0120-a3c2-4a29-b3aa-93f1bd339e35 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:47.597022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:47.661231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f33d06-b9bd-4e53-b912-119cdebeecc4 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:47.683377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cecc0120-a3c2-4a29-b3aa-93f1bd339e35 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:47.696323 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ecece-9d20-49a0-aa51-28ea9e15794d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:47.697000 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ecece-9d20-49a0-aa51-28ea9e15794d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:47.697290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ecece-9d20-49a0-aa51-28ea9e15794d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:47.697543 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555ecece-9d20-49a0-aa51-28ea9e15794d None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:47.698278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ecece-9d20-49a0-aa51-28ea9e15794d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:47.698593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ecece-9d20-49a0-aa51-28ea9e15794d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:47.700066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555ecece-9d20-49a0-aa51-28ea9e15794d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:47.711414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 365/743] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:47.712223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ecece-9d20-49a0-aa51-28ea9e15794d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:47.721579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c88b8-54c1-48f0-bece-73fcedfd3b4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:47.722259 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c88b8-54c1-48f0-bece-73fcedfd3b4b None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:47.722552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c88b8-54c1-48f0-bece-73fcedfd3b4b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:47.722839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751c88b8-54c1-48f0-bece-73fcedfd3b4b None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:47.723578 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c88b8-54c1-48f0-bece-73fcedfd3b4b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:47.725002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751c88b8-54c1-48f0-bece-73fcedfd3b4b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:47.740862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 379/744] 10.222.0.85 () {60 vars in 1141 bytes} [Wed May 25 15:54:47 2022] POST /identity/v3/services => generated 251 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:54:47.741576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c88b8-54c1-48f0-bece-73fcedfd3b4b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:47.815976 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:47.820365 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:47.822911 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555ecece-9d20-49a0-aa51-28ea9e15794d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:47.873002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd526c0c-2012-4c59-b235-4037fdee80f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:47.873974 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd526c0c-2012-4c59-b235-4037fdee80f9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:47.874265 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd526c0c-2012-4c59-b235-4037fdee80f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:47.874550 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd526c0c-2012-4c59-b235-4037fdee80f9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:47.876623 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751c88b8-54c1-48f0-bece-73fcedfd3b4b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:47.919146 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70967685-7a6f-459d-aca3-edc83daed466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:47.919813 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70967685-7a6f-459d-aca3-edc83daed466 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:47.920620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70967685-7a6f-459d-aca3-edc83daed466 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:47.920912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70967685-7a6f-459d-aca3-edc83daed466 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:47.921658 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70967685-7a6f-459d-aca3-edc83daed466 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:47.924026 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70967685-7a6f-459d-aca3-edc83daed466 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:47.940583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 380/745] 10.222.0.85 () {60 vars in 1141 bytes} [Wed May 25 15:54:47 2022] POST /identity/v3/services => generated 240 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:54:47.941428 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70967685-7a6f-459d-aca3-edc83daed466 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:47.962109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd526c0c-2012-4c59-b235-4037fdee80f9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:47.963647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd526c0c-2012-4c59-b235-4037fdee80f9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:48.084432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:48.087084 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70967685-7a6f-459d-aca3-edc83daed466 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:48.103175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 366/746] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:47 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) May 25 15:54:48.104607 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd526c0c-2012-4c59-b235-4037fdee80f9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:48.104923 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:48.132544 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea4a8c-fd54-4868-9d8f-0e675d05fb01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:48.133553 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea4a8c-fd54-4868-9d8f-0e675d05fb01 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:48.133977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea4a8c-fd54-4868-9d8f-0e675d05fb01 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:48.134281 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fea4a8c-fd54-4868-9d8f-0e675d05fb01 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:48.145021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea4a8c-fd54-4868-9d8f-0e675d05fb01 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:48.146802 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fea4a8c-fd54-4868-9d8f-0e675d05fb01 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:48.162516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-7fea4a8c-fd54-4868-9d8f-0e675d05fb01 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 15:54:48.162516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:48.162516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:48.162516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:48.162516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:48.162516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:48.162516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:48.162516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:48.162516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:48.162516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:48.162516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:48.162516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:48.162516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:48.162516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:48.162516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:48.162516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:48.163739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:48.164991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:48.164991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:48.164991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:48.164991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:48.164991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:48.164991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:48.164991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:48.164991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:48.164991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 15:54:48.164991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:48.165665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 381/747] 10.222.0.85 () {56 vars in 1117 bytes} [Wed May 25 15:54:47 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:48.166303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea4a8c-fd54-4868-9d8f-0e675d05fb01 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:48.234854 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd526c0c-2012-4c59-b235-4037fdee80f9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:48.276229 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28439ff-2f32-4522-aa5e-506c1f1001fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:48.277438 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28439ff-2f32-4522-aa5e-506c1f1001fd None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:48.278091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28439ff-2f32-4522-aa5e-506c1f1001fd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:48.278380 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28439ff-2f32-4522-aa5e-506c1f1001fd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:48.283463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fea4a8c-fd54-4868-9d8f-0e675d05fb01 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:48.287533 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28439ff-2f32-4522-aa5e-506c1f1001fd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:48.289026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28439ff-2f32-4522-aa5e-506c1f1001fd None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:48.309429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-a28439ff-2f32-4522-aa5e-506c1f1001fd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 15:54:48.309429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:48.309429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:48.309429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:48.309429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:48.309429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:48.309429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:48.309429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:48.309429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:48.309429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:48.309429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:48.309429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:48.309429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:48.309429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:48.309429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:48.309429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:48.310553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:48.311500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:48.311500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:48.311500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:48.311500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:48.311500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:48.311500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:48.311500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:48.311500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:48.311500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 15:54:48.311500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:48.311500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 367/748] 10.222.0.85 () {56 vars in 1093 bytes} [Wed May 25 15:54:48 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:48.312122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28439ff-2f32-4522-aa5e-506c1f1001fd None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:48.323981 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191e7ef-ce18-44f8-89a0-c3dc5bc405f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:48.325089 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191e7ef-ce18-44f8-89a0-c3dc5bc405f4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:48.325619 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191e7ef-ce18-44f8-89a0-c3dc5bc405f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:48.326043 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2191e7ef-ce18-44f8-89a0-c3dc5bc405f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:48.338679 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2191e7ef-ce18-44f8-89a0-c3dc5bc405f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:48.340630 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2191e7ef-ce18-44f8-89a0-c3dc5bc405f4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:48.359855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2191e7ef-ce18-44f8-89a0-c3dc5bc405f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:48.359855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:48.359855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:48.359855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:48.359855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:48.359855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:48.359855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:48.359855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:48.359855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:48.359855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:48.359855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:48.359855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:48.359855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:48.359855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:48.359855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:48.361020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:48.361020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:48.361020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:48.361020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:48.361020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:48.361020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:48.361020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:48.361020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:48.361020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:48.361020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:48.361020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:48.361020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:48.361020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:48.361020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:48.361020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:48.361020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:48.361020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:48.362103 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:48.362103 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:48.362103 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:48.362103 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:48.364139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 382/749] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:48.365071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191e7ef-ce18-44f8-89a0-c3dc5bc405f4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:48.370292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:48.394527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:48.421103 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28439ff-2f32-4522-aa5e-506c1f1001fd None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:48.456225 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597ed1d-8de3-41c6-bdac-a726d8644dc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:48.457259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597ed1d-8de3-41c6-bdac-a726d8644dc2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:48.457815 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597ed1d-8de3-41c6-bdac-a726d8644dc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:48.457933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a597ed1d-8de3-41c6-bdac-a726d8644dc2 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:48.458729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597ed1d-8de3-41c6-bdac-a726d8644dc2 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:48.459042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597ed1d-8de3-41c6-bdac-a726d8644dc2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:48.460623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a597ed1d-8de3-41c6-bdac-a726d8644dc2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:48.476901 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2191e7ef-ce18-44f8-89a0-c3dc5bc405f4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:48.480246 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 368/750] 10.222.0.85 () {56 vars in 1149 bytes} [Wed May 25 15:54:48 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:48.481230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597ed1d-8de3-41c6-bdac-a726d8644dc2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:48.513121 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b57b3f-1ce6-4a19-b746-995ae88680c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:48.514115 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b57b3f-1ce6-4a19-b746-995ae88680c0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:48.514580 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b57b3f-1ce6-4a19-b746-995ae88680c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:48.515035 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b57b3f-1ce6-4a19-b746-995ae88680c0 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:48.516110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b57b3f-1ce6-4a19-b746-995ae88680c0 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:48.516777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b57b3f-1ce6-4a19-b746-995ae88680c0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:48.518436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b57b3f-1ce6-4a19-b746-995ae88680c0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:48.541192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 383/751] 10.222.0.85 () {56 vars in 1133 bytes} [Wed May 25 15:54:48 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:48.542356 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b57b3f-1ce6-4a19-b746-995ae88680c0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:48.588352 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a597ed1d-8de3-41c6-bdac-a726d8644dc2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:48.602556 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:48.630189 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccad944-59fc-4706-96e5-f4fa2d35c056 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:48.631109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccad944-59fc-4706-96e5-f4fa2d35c056 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:48.631546 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccad944-59fc-4706-96e5-f4fa2d35c056 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:48.632032 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bccad944-59fc-4706-96e5-f4fa2d35c056 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:48.632483 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:48.633069 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccad944-59fc-4706-96e5-f4fa2d35c056 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:48.633528 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccad944-59fc-4706-96e5-f4fa2d35c056 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:48.635122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bccad944-59fc-4706-96e5-f4fa2d35c056 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:48.652460 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 369/752] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:48.652687 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b57b3f-1ce6-4a19-b746-995ae88680c0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:48.653742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccad944-59fc-4706-96e5-f4fa2d35c056 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:48.688803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3e9b9-a6fa-40c8-adb9-88f9b2bfde0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:48.689465 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3e9b9-a6fa-40c8-adb9-88f9b2bfde0d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:48.689752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3e9b9-a6fa-40c8-adb9-88f9b2bfde0d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:48.690031 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ba3e9b9-a6fa-40c8-adb9-88f9b2bfde0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:48.758677 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bccad944-59fc-4706-96e5-f4fa2d35c056 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:48.760445 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3e9b9-a6fa-40c8-adb9-88f9b2bfde0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:48.762099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ba3e9b9-a6fa-40c8-adb9-88f9b2bfde0d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:48.794308 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc91bc7-d1d0-49cb-bbad-66dbc7f3ee4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:48.795450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc91bc7-d1d0-49cb-bbad-66dbc7f3ee4e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:48.795912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc91bc7-d1d0-49cb-bbad-66dbc7f3ee4e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:48.796349 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc91bc7-d1d0-49cb-bbad-66dbc7f3ee4e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:48.807770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc91bc7-d1d0-49cb-bbad-66dbc7f3ee4e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:48.809594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbc91bc7-d1d0-49cb-bbad-66dbc7f3ee4e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:48.827562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-fbc91bc7-d1d0-49cb-bbad-66dbc7f3ee4e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:48.827562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:48.827562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:48.827562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:48.827562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:48.827562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:48.827562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:48.827562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:48.827562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:48.827562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:48.827562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:48.827562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:48.827562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:48.827562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:48.827562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:48.828718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:48.828718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:48.828718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:48.828718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:48.828718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:48.828718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:48.828718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:48.828718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:48.828718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:48.828718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:48.828718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:48.828718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:48.828718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:48.828718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:48.828718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:48.828718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:48.828718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:48.829860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:48.829860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:48.829860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:48.829860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:48.832129 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 370/753] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:48.833145 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc91bc7-d1d0-49cb-bbad-66dbc7f3ee4e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:48.844625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:48.859532 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:48.865709 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 384/754] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:48 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:54:48.866641 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3e9b9-a6fa-40c8-adb9-88f9b2bfde0d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:48.962782 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbc91bc7-d1d0-49cb-bbad-66dbc7f3ee4e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:49.000047 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ba3e9b9-a6fa-40c8-adb9-88f9b2bfde0d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:49.005854 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11eb72-c0e0-445f-b5f5-15cc1bfc1042 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:49.006737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11eb72-c0e0-445f-b5f5-15cc1bfc1042 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:49.007111 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11eb72-c0e0-445f-b5f5-15cc1bfc1042 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:49.007462 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc11eb72-c0e0-445f-b5f5-15cc1bfc1042 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:49.016413 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11eb72-c0e0-445f-b5f5-15cc1bfc1042 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:49.018701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc11eb72-c0e0-445f-b5f5-15cc1bfc1042 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:49.038150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-dc11eb72-c0e0-445f-b5f5-15cc1bfc1042 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:49.038150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:49.038150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:49.038150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:49.038150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:49.038150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:49.038150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:49.038150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:49.038150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:49.038150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:49.038150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:49.038150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:49.038150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:49.038150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:49.038150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:49.039298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:49.039298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:49.039298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:49.039298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:49.039298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:49.039298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:49.039298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:49.039298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:49.039298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:49.039298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:49.039298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:49.039298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:49.039298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:49.039298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:49.039298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:49.039298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:49.039298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:49.040480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:49.040480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:49.040480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:49.040480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:49.040480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:49.040480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:49.040480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:49.040480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:49.045758 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 371/755] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:49.045758 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11eb72-c0e0-445f-b5f5-15cc1bfc1042 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:49.049307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fcd14-73b3-4e74-b4bf-884023d07a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:49.050199 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fcd14-73b3-4e74-b4bf-884023d07a56 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:49.050638 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fcd14-73b3-4e74-b4bf-884023d07a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:49.051068 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fcd14-73b3-4e74-b4bf-884023d07a56 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:49.052005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fcd14-73b3-4e74-b4bf-884023d07a56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:49.052991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fcd14-73b3-4e74-b4bf-884023d07a56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:49.055121 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fcd14-73b3-4e74-b4bf-884023d07a56 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:49.070445 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 385/756] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:49.071389 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fcd14-73b3-4e74-b4bf-884023d07a56 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:49.115853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:49.135551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:49.156158 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc11eb72-c0e0-445f-b5f5-15cc1bfc1042 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:49.178107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fcd14-73b3-4e74-b4bf-884023d07a56 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:49.192011 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67be782-918a-481b-a9bf-014d1d4d97ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:49.193140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67be782-918a-481b-a9bf-014d1d4d97ad None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:49.193577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67be782-918a-481b-a9bf-014d1d4d97ad None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:49.194002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67be782-918a-481b-a9bf-014d1d4d97ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:49.205181 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67be782-918a-481b-a9bf-014d1d4d97ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:49.207201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67be782-918a-481b-a9bf-014d1d4d97ad None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:49.213350 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414c7da-816f-449f-a819-e7c80f538b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:49.214021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414c7da-816f-449f-a819-e7c80f538b06 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:49.214305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414c7da-816f-449f-a819-e7c80f538b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:49.214586 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9414c7da-816f-449f-a819-e7c80f538b06 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:49.215628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414c7da-816f-449f-a819-e7c80f538b06 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:49.215749 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414c7da-816f-449f-a819-e7c80f538b06 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:49.217239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9414c7da-816f-449f-a819-e7c80f538b06 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:49.227034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-e67be782-918a-481b-a9bf-014d1d4d97ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:49.227034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:49.227034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:49.227034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:49.227034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:49.227034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:49.227034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:49.227034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:49.227034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:49.227034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:49.227034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:49.227034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:49.227034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:49.227034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:49.227034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:49.229330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:49.229330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:49.229330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:49.229330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:49.235010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 386/757] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:49.235010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414c7da-816f-449f-a819-e7c80f538b06 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:49.236237 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 372/758] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:49.237734 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67be782-918a-481b-a9bf-014d1d4d97ad None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:49.339289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9414c7da-816f-449f-a819-e7c80f538b06 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:49.340998 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:49.342782 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67be782-918a-481b-a9bf-014d1d4d97ad None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:49.375989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4ca51-c2e0-4533-891f-a25d335abac0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:49.376718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4ca51-c2e0-4533-891f-a25d335abac0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:49.377003 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4ca51-c2e0-4533-891f-a25d335abac0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:49.377282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e4ca51-c2e0-4533-891f-a25d335abac0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:49.378490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1573ba7-2570-4e19-b2c2-4ae39b8f7538 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:49.379324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1573ba7-2570-4e19-b2c2-4ae39b8f7538 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:49.379755 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1573ba7-2570-4e19-b2c2-4ae39b8f7538 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:49.380207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1573ba7-2570-4e19-b2c2-4ae39b8f7538 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:49.381189 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1573ba7-2570-4e19-b2c2-4ae39b8f7538 None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=77fda27641c84fccb68a5cdaf4e8dc4b) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:49.381659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1573ba7-2570-4e19-b2c2-4ae39b8f7538 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:49.383274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1573ba7-2570-4e19-b2c2-4ae39b8f7538 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:49.384234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4ca51-c2e0-4533-891f-a25d335abac0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:49.385661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:49.386834 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e4ca51-c2e0-4533-891f-a25d335abac0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:49.399917 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 373/759] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:54:49 2022] GET /identity/v3/role_assignments?user.id=77fda27641c84fccb68a5cdaf4e8dc4b&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:49.401265 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1573ba7-2570-4e19-b2c2-4ae39b8f7538 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:49.402777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d4e4ca51-c2e0-4533-891f-a25d335abac0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:49.402777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:49.402777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:49.402777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:49.402777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:49.402777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:49.402777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:49.402777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:49.402777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:49.402777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:49.402777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:49.402777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:49.402777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:49.402777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:49.402777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:49.403640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:49.403640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:49.403640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:49.403640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:49.403640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:49.403640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:49.403640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:49.403640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:49.403640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:49.403640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:49.403640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:49.403640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:49.403640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:49.403640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:49.403640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:49.403640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:49.403640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:49.404940 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:49.404940 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:49.404940 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:49.404940 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:49.404940 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:49.404940 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:49.404940 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:49.404940 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:49.404940 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 387/760] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:49.409880 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4ca51-c2e0-4533-891f-a25d335abac0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:49.533721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e4ca51-c2e0-4533-891f-a25d335abac0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:49.575138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc230c64-105c-4a23-afc2-653be1bc6807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:49.576010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc230c64-105c-4a23-afc2-653be1bc6807 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:49.576297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc230c64-105c-4a23-afc2-653be1bc6807 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:49.576577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc230c64-105c-4a23-afc2-653be1bc6807 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:49.577460 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc230c64-105c-4a23-afc2-653be1bc6807 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:49.578039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc230c64-105c-4a23-afc2-653be1bc6807 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:49.580064 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc230c64-105c-4a23-afc2-653be1bc6807 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:49.595437 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 388/761] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:49.601399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc230c64-105c-4a23-afc2-653be1bc6807 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:49.631329 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:49.694007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1573ba7-2570-4e19-b2c2-4ae39b8f7538 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:49.726333 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc230c64-105c-4a23-afc2-653be1bc6807 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:49.741665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2101140-3060-4597-9d0c-a0006f1c755c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:49.743095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2101140-3060-4597-9d0c-a0006f1c755c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:49.743620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2101140-3060-4597-9d0c-a0006f1c755c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:49.744066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2101140-3060-4597-9d0c-a0006f1c755c None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:49.745089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2101140-3060-4597-9d0c-a0006f1c755c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:49.745835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2101140-3060-4597-9d0c-a0006f1c755c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:49.747453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2101140-3060-4597-9d0c-a0006f1c755c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:49.763716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 374/762] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 363 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:49.768335 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19748e1-9ca4-49ac-9a43-6acb5d9e4115 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:49.769415 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2101140-3060-4597-9d0c-a0006f1c755c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:49.770870 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19748e1-9ca4-49ac-9a43-6acb5d9e4115 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:49.771657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19748e1-9ca4-49ac-9a43-6acb5d9e4115 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:49.776873 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d19748e1-9ca4-49ac-9a43-6acb5d9e4115 None admin] PATH_INFO: `/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/4751abc9f67947b3ad17ba2b78ce2c12/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:49.788715 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19748e1-9ca4-49ac-9a43-6acb5d9e4115 None admin] RBAC: Authorizing `identity:create_grant(project_id=5584ebdaf133444ca69efc5554bf48e9, user_id=4751abc9f67947b3ad17ba2b78ce2c12, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:49.790209 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d19748e1-9ca4-49ac-9a43-6acb5d9e4115 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:49.836927 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:49.842637 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 389/763] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:54:49 2022] PUT /identity/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/4751abc9f67947b3ad17ba2b78ce2c12/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:54:49.899700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2101140-3060-4597-9d0c-a0006f1c755c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:49.945930 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8537df9e-d905-4e50-b1d0-352389caf6e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:49.947093 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537df9e-d905-4e50-b1d0-352389caf6e6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:49.947558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537df9e-d905-4e50-b1d0-352389caf6e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:49.948050 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8537df9e-d905-4e50-b1d0-352389caf6e6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:49.956981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537df9e-d905-4e50-b1d0-352389caf6e6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:49.958797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8537df9e-d905-4e50-b1d0-352389caf6e6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:49.974126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-8537df9e-d905-4e50-b1d0-352389caf6e6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:49.974126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:49.974126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:49.974126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:49.974126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:49.974126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:49.974126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:49.974126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:49.974126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:49.974126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:49.974126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:49.974126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:49.974126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:49.974126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:49.974126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:49.975685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:49.975685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:49.975685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:49.975685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:49.975685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:49.975685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:49.975685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:49.975685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:49.975685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:49.975685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:49.975685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:49.975685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:49.975685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:49.975685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:49.975685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:49.975685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:49.975685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:49.976802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:49.976802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:49.976802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:49.976802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:49.976802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:49.976802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:49.976802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:49.976802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:49.978941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 375/764] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:49.984451 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19748e1-9ca4-49ac-9a43-6acb5d9e4115 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:50.117390 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6415f4-1d60-46ee-99a5-be32bdc2699b None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:50.117947 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6415f4-1d60-46ee-99a5-be32bdc2699b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:50.118376 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc6415f4-1d60-46ee-99a5-be32bdc2699b None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:50.119956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 376/765] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:50.125897 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4ea7f-857b-41f2-aef6-260640143e98 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:50.126341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4ea7f-857b-41f2-aef6-260640143e98 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:50.126733 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fa4ea7f-857b-41f2-aef6-260640143e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:50.139852 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:50.158703 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-0fa4ea7f-857b-41f2-aef6-260640143e98 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:50.159986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0fa4ea7f-857b-41f2-aef6-260640143e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:50.165947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d19748e1-9ca4-49ac-9a43-6acb5d9e4115 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:50.214732 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f4eaf-fd36-4e85-bd1c-73071bfc5725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:50.215796 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f4eaf-fd36-4e85-bd1c-73071bfc5725 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:50.216167 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f4eaf-fd36-4e85-bd1c-73071bfc5725 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:50.216507 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b4f4eaf-fd36-4e85-bd1c-73071bfc5725 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:50.217369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f4eaf-fd36-4e85-bd1c-73071bfc5725 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:50.217986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f4eaf-fd36-4e85-bd1c-73071bfc5725 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:50.220007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b4f4eaf-fd36-4e85-bd1c-73071bfc5725 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:50.236410 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 390/766] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:50.241917 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f4eaf-fd36-4e85-bd1c-73071bfc5725 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:50.244075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 377/767] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:54:50.269312 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4ea7f-857b-41f2-aef6-260640143e98 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:50.322605 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:50.349727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b4f4eaf-fd36-4e85-bd1c-73071bfc5725 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:50.385188 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ae91b3-5d07-42ca-897d-7e09a772788b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:50.385945 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae91b3-5d07-42ca-897d-7e09a772788b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:50.386300 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae91b3-5d07-42ca-897d-7e09a772788b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:50.386695 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ae91b3-5d07-42ca-897d-7e09a772788b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:50.387583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae91b3-5d07-42ca-897d-7e09a772788b None admin] RBAC: Adding query filter params (role.id=83bf7d9c29d4464dab8030f6d706cd6d, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=066de1f11f724eee9df0377beb58d92c) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:50.391840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae91b3-5d07-42ca-897d-7e09a772788b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:50.391840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ae91b3-5d07-42ca-897d-7e09a772788b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:50.393348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fa4ea7f-857b-41f2-aef6-260640143e98 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:50.395673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 391/768] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:54:50 2022] GET /identity/v3/role_assignments?user.id=066de1f11f724eee9df0377beb58d92c&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=83bf7d9c29d4464dab8030f6d706cd6d => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:50.421375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73960cd5-d832-4df6-9e86-a313f361f9d0 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:50.421557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73960cd5-d832-4df6-9e86-a313f361f9d0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:50.421759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73960cd5-d832-4df6-9e86-a313f361f9d0 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:50.423500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 392/769] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:54:50.429449 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da9985-7b25-4c3c-a55d-a79fb3a8d9ed None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:50.429684 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da9985-7b25-4c3c-a55d-a79fb3a8d9ed None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:50.429983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01da9985-7b25-4c3c-a55d-a79fb3a8d9ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:50.434818 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c132c-0a71-4182-b1ea-5ab563f6aaee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:50.436084 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c132c-0a71-4182-b1ea-5ab563f6aaee None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:50.436677 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c132c-0a71-4182-b1ea-5ab563f6aaee None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:50.437160 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902c132c-0a71-4182-b1ea-5ab563f6aaee None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:50.438044 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c132c-0a71-4182-b1ea-5ab563f6aaee None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:50.440054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902c132c-0a71-4182-b1ea-5ab563f6aaee None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:50.454207 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:50.455865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-902c132c-0a71-4182-b1ea-5ab563f6aaee None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 15:54:50.455865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:50.455865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:54:50.455865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:54:50.455865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:54:50.455865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:54:50.455865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:54:50.455865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:54:50.455865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 15:54:50.455865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:50.455865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:54:50.455865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:50.455865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:50.455865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:50.455865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:50.455865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:50.457144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:50.457144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:50.457144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:50.457144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:50.457144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:50.457144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:50.457144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:50.457144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:54:50.457144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:54:50.457144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:54:50.457144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:54:50.457144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:50.457144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:50.457144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:50.457144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:50.457144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:50.458023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:50.458023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:50.458023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:50.458023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:50.458023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:50.458023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:50.458023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:50.458023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:50.458023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:50.458023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:50.458023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:50.458023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:54:50.458023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:54:50.458023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 15:54:50.458023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:50.461115 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 378/770] 10.222.0.85 () {56 vars in 1129 bytes} [Wed May 25 15:54:50 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:54:50.468410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c132c-0a71-4182-b1ea-5ab563f6aaee None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:50.470429 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-01da9985-7b25-4c3c-a55d-a79fb3a8d9ed None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:50.471624 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-01da9985-7b25-4c3c-a55d-a79fb3a8d9ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:50.605341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902c132c-0a71-4182-b1ea-5ab563f6aaee None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:50.610617 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 393/771] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:54:50.633732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:50.641014 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da9985-7b25-4c3c-a55d-a79fb3a8d9ed None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:50.643266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67583ac7-23aa-4ae2-9a35-8d7770c83528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:50.643994 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67583ac7-23aa-4ae2-9a35-8d7770c83528 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:50.644286 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67583ac7-23aa-4ae2-9a35-8d7770c83528 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:50.644569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67583ac7-23aa-4ae2-9a35-8d7770c83528 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:50.645347 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67583ac7-23aa-4ae2-9a35-8d7770c83528 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:50.645663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67583ac7-23aa-4ae2-9a35-8d7770c83528 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:50.647096 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67583ac7-23aa-4ae2-9a35-8d7770c83528 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:50.653462 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 379/772] 10.222.0.85 () {56 vars in 1124 bytes} [Wed May 25 15:54:50 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:50.659209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67583ac7-23aa-4ae2-9a35-8d7770c83528 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:50.770924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67583ac7-23aa-4ae2-9a35-8d7770c83528 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:50.806522 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae2c92-b43f-4fdd-8ffe-22ca47ac7246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:50.807736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae2c92-b43f-4fdd-8ffe-22ca47ac7246 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:50.808056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae2c92-b43f-4fdd-8ffe-22ca47ac7246 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:50.808336 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ae2c92-b43f-4fdd-8ffe-22ca47ac7246 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:50.809101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae2c92-b43f-4fdd-8ffe-22ca47ac7246 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:50.809420 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae2c92-b43f-4fdd-8ffe-22ca47ac7246 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:50.810819 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ae2c92-b43f-4fdd-8ffe-22ca47ac7246 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:50.816649 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 380/773] 10.222.0.85 () {56 vars in 1124 bytes} [Wed May 25 15:54:50 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:50.822225 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae2c92-b43f-4fdd-8ffe-22ca47ac7246 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:50.880214 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:50.916096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01da9985-7b25-4c3c-a55d-a79fb3a8d9ed None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:50.987179 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ae2c92-b43f-4fdd-8ffe-22ca47ac7246 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:50.996581 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a496da8-d9c0-4df8-a144-ad1784092b63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:50.997931 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a496da8-d9c0-4df8-a144-ad1784092b63 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:51.001673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a496da8-d9c0-4df8-a144-ad1784092b63 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:51.002264 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a496da8-d9c0-4df8-a144-ad1784092b63 None admin] PATH_INFO: `/v3/services/network` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:51.003170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a496da8-d9c0-4df8-a144-ad1784092b63 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:51.004847 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a496da8-d9c0-4df8-a144-ad1784092b63 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:51.022616 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a097a-f8d8-49ca-9d27-6a8231d70799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:51.023204 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a097a-f8d8-49ca-9d27-6a8231d70799 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:51.023328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:51.023490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a097a-f8d8-49ca-9d27-6a8231d70799 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:51.023767 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400a097a-f8d8-49ca-9d27-6a8231d70799 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:51.024593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a097a-f8d8-49ca-9d27-6a8231d70799 None admin] RBAC: Adding query filter params (interface=public, service_id=d1a855cf26d84944b4d9b68e6b56b42e, region_id=RegionOne) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:51.024930 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a097a-f8d8-49ca-9d27-6a8231d70799 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:51.026586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400a097a-f8d8-49ca-9d27-6a8231d70799 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:51.038878 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 381/774] 10.222.0.85 () {56 vars in 1251 bytes} [Wed May 25 15:54:50 2022] GET /identity/v3/endpoints?service_id=d1a855cf26d84944b4d9b68e6b56b42e&interface=public®ion_id=RegionOne => generated 194 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:51.039582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-5a496da8-d9c0-4df8-a144-ad1784092b63 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 15:54:51.039582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:51.039582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:54:51.039582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:54:51.039582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:54:51.039582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:54:51.039582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:54:51.039582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:54:51.039582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 15:54:51.039582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:51.039582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:54:51.039582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:51.039582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:51.039582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:51.039582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:51.039582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:51.039582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:51.040570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:51.040570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:51.040570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:51.040570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:51.040570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:51.040570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:51.040570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:54:51.040570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:54:51.040570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:54:51.040570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:54:51.040570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:51.040570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:51.040570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:51.040570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:51.040570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:51.040570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:51.041430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:51.041430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:51.041430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:51.041430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:51.041430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:51.041430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:51.041430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:51.041430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:51.041430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:51.041430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:51.041430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:54:51.041430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:54:51.041430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 15:54:51.041430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:51.041430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 394/775] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:50 2022] GET /identity/v3/services/network => generated 88 bytes in 399 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:54:51.042249 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a097a-f8d8-49ca-9d27-6a8231d70799 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:51.049541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a496da8-d9c0-4df8-a144-ad1784092b63 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:51.219225 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400a097a-f8d8-49ca-9d27-6a8231d70799 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:51.224831 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:51.249537 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a496da8-d9c0-4df8-a144-ad1784092b63 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:51.283099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed50d07-ecfd-4f64-a02c-b3502004c4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:51.285772 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed50d07-ecfd-4f64-a02c-b3502004c4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:51.286061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed50d07-ecfd-4f64-a02c-b3502004c4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:51.286343 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed50d07-ecfd-4f64-a02c-b3502004c4a0 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:51.287037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed50d07-ecfd-4f64-a02c-b3502004c4a0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:51.288459 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ed50d07-ecfd-4f64-a02c-b3502004c4a0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:51.297711 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 382/776] 10.222.0.85 () {56 vars in 1087 bytes} [Wed May 25 15:54:51 2022] GET /identity/v3/services => generated 772 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:51.326403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211b76c-fc54-4f1d-8dcb-c513744e9ca6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:51.327206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211b76c-fc54-4f1d-8dcb-c513744e9ca6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:51.327497 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211b76c-fc54-4f1d-8dcb-c513744e9ca6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:51.327778 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e211b76c-fc54-4f1d-8dcb-c513744e9ca6 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:51.328572 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211b76c-fc54-4f1d-8dcb-c513744e9ca6 None admin] RBAC: Adding query filter params (name=network) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:51.328888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211b76c-fc54-4f1d-8dcb-c513744e9ca6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:51.330355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e211b76c-fc54-4f1d-8dcb-c513744e9ca6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:51.336757 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 395/777] 10.222.0.85 () {56 vars in 1112 bytes} [Wed May 25 15:54:51 2022] GET /identity/v3/services?name=network => generated 124 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:51.342786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed50d07-ecfd-4f64-a02c-b3502004c4a0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:51.489777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed50d07-ecfd-4f64-a02c-b3502004c4a0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:51.535334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae29988-6413-413d-b7c4-299c7cfbd5f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:51.536395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae29988-6413-413d-b7c4-299c7cfbd5f8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:51.536880 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae29988-6413-413d-b7c4-299c7cfbd5f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:51.537349 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae29988-6413-413d-b7c4-299c7cfbd5f8 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:51.538288 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae29988-6413-413d-b7c4-299c7cfbd5f8 None admin] RBAC: Adding query filter params (type=network) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:51.538792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae29988-6413-413d-b7c4-299c7cfbd5f8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:51.540409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae29988-6413-413d-b7c4-299c7cfbd5f8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:51.550636 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 383/778] 10.222.0.85 () {56 vars in 1112 bytes} [Wed May 25 15:54:51 2022] GET /identity/v3/services?type=network => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:51.556767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211b76c-fc54-4f1d-8dcb-c513744e9ca6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:51.634217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:51.677874 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e211b76c-fc54-4f1d-8dcb-c513744e9ca6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:51.714751 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dce58a-af50-42c8-8df0-e421d3c7e80c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:51.715481 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dce58a-af50-42c8-8df0-e421d3c7e80c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:51.715836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dce58a-af50-42c8-8df0-e421d3c7e80c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:51.716141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dce58a-af50-42c8-8df0-e421d3c7e80c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:51.716956 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dce58a-af50-42c8-8df0-e421d3c7e80c None admin] RBAC: Adding query filter params (interface=public, service_id=3a747bfbe37f4efa9c75ad27d034bc4d, region_id=RegionOne) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:51.717280 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dce58a-af50-42c8-8df0-e421d3c7e80c None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:51.718687 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dce58a-af50-42c8-8df0-e421d3c7e80c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:51.727160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 396/779] 10.222.0.85 () {56 vars in 1251 bytes} [Wed May 25 15:54:51 2022] GET /identity/v3/endpoints?service_id=3a747bfbe37f4efa9c75ad27d034bc4d&interface=public®ion_id=RegionOne => generated 194 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:51.732923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae29988-6413-413d-b7c4-299c7cfbd5f8 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:51.787031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:51.875568 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae29988-6413-413d-b7c4-299c7cfbd5f8 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:51.922982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce3c01e-4ca4-4932-b26e-3d9bcb530349 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:51.923676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3c01e-4ca4-4932-b26e-3d9bcb530349 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:51.924000 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3c01e-4ca4-4932-b26e-3d9bcb530349 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:51.924286 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce3c01e-4ca4-4932-b26e-3d9bcb530349 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:51.924992 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce3c01e-4ca4-4932-b26e-3d9bcb530349 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:51.926408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce3c01e-4ca4-4932-b26e-3d9bcb530349 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:51.937409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 384/780] 10.222.0.85 () {56 vars in 1087 bytes} [Wed May 25 15:54:51 2022] GET /identity/v3/services => generated 772 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:52.231005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4588d-fd74-4173-a0b2-e5e9b9db7a9d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:52.231512 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4588d-fd74-4173-a0b2-e5e9b9db7a9d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:52.231948 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de4588d-fd74-4173-a0b2-e5e9b9db7a9d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:52.233699 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 397/781] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:52.239045 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2d211-50dc-43f3-bed1-9c696721be9c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:52.239505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2d211-50dc-43f3-bed1-9c696721be9c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:52.239614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff2d211-50dc-43f3-bed1-9c696721be9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:52.262582 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-bff2d211-50dc-43f3-bed1-9c696721be9c None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:52.263798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-bff2d211-50dc-43f3-bed1-9c696721be9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:52.356141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 385/782] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:54:52.388957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4588d-fd74-4173-a0b2-e5e9b9db7a9d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:52.545806 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de4588d-fd74-4173-a0b2-e5e9b9db7a9d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:52.565526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96242551-a1be-44c1-bca8-d971a75cd337 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:52.565862 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96242551-a1be-44c1-bca8-d971a75cd337 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:52.566100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96242551-a1be-44c1-bca8-d971a75cd337 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:52.567839 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 386/783] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:52.573145 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f6412-a83a-4a5d-b4b9-ad7a5ee917aa None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:52.573443 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f6412-a83a-4a5d-b4b9-ad7a5ee917aa None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:52.573732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca6f6412-a83a-4a5d-b4b9-ad7a5ee917aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:52.580809 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:52.589435 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1b1cc-812c-405a-bba7-c7eb4e11c020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:52.590481 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1b1cc-812c-405a-bba7-c7eb4e11c020 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:52.590778 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1b1cc-812c-405a-bba7-c7eb4e11c020 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:52.591065 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c1b1cc-812c-405a-bba7-c7eb4e11c020 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:52.602613 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c1b1cc-812c-405a-bba7-c7eb4e11c020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:52.610584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-90c1b1cc-812c-405a-bba7-c7eb4e11c020 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:52.610584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:52.610584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:52.610584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:52.610584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:52.610584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:52.610584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:52.610584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:52.610584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:52.610584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:52.610584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:52.610584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:52.610584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:52.610584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:52.610584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:52.610584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:52.611506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:52.611506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:52.611506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:52.611506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:52.611506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:52.611506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:52.611506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:52.611506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:52.611506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:52.611506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:52.611506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:52.611506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:52.611506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:52.611506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:52.611506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:52.611506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:52.612374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:52.612374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:52.612374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:52.612374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:52.612374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:52.612374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:52.612374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:52.612374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:52.612374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:52.612374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:52.612374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:52.612374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:52.613004 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ca6f6412-a83a-4a5d-b4b9-ad7a5ee917aa None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:52.613077 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 398/784] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:54:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:52.613143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ca6f6412-a83a-4a5d-b4b9-ad7a5ee917aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:52.620505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1b1cc-812c-405a-bba7-c7eb4e11c020 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:52.672056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:52.736330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c1b1cc-812c-405a-bba7-c7eb4e11c020 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:52.774737 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29378586-e348-43ee-ba45-ea2f61f5f219 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:52.775651 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378586-e348-43ee-ba45-ea2f61f5f219 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:52.775997 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378586-e348-43ee-ba45-ea2f61f5f219 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:52.776284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29378586-e348-43ee-ba45-ea2f61f5f219 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:52.777061 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29378586-e348-43ee-ba45-ea2f61f5f219 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:52.777447 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29378586-e348-43ee-ba45-ea2f61f5f219 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:52.779069 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29378586-e348-43ee-ba45-ea2f61f5f219 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:52.789081 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 399/785] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:52.800911 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29378586-e348-43ee-ba45-ea2f61f5f219 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:52.811199 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 387/786] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:52 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:54:52.846181 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f6412-a83a-4a5d-b4b9-ad7a5ee917aa None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:52.878291 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:52.933731 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29378586-e348-43ee-ba45-ea2f61f5f219 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:52.969671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d52c5035-eaa9-4cfa-aaa8-6b3f780e08de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:52.970650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c5035-eaa9-4cfa-aaa8-6b3f780e08de None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:52.970942 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c5035-eaa9-4cfa-aaa8-6b3f780e08de None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:52.971225 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d52c5035-eaa9-4cfa-aaa8-6b3f780e08de None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:52.983129 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c5035-eaa9-4cfa-aaa8-6b3f780e08de None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:52.984870 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d52c5035-eaa9-4cfa-aaa8-6b3f780e08de None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:53.004097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d52c5035-eaa9-4cfa-aaa8-6b3f780e08de None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:53.004097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:53.004097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:53.004097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:53.004097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:53.004097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:53.004097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:53.004097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:53.004097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:53.004097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:53.004097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:53.004097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:53.004097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:53.004097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:53.004097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:53.005040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:53.005040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:53.005040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:53.005040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:53.005040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:53.005040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:53.005040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:53.005040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:53.005040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:53.005040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:53.005040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:53.005040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:53.005040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:53.005040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:53.005040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:53.005040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:53.005040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:53.005924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:53.005924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:53.005924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:53.005924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:53.005924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 400/787] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:53.007194 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c0ff9-0222-4531-b9d0-0c5da3599958 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:53.010403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c0ff9-0222-4531-b9d0-0c5da3599958 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:53.010403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c0ff9-0222-4531-b9d0-0c5da3599958 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:53.010403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 401/788] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:53.011326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c0ff9-0222-4531-b9d0-0c5da3599958 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:53.025965 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca6f6412-a83a-4a5d-b4b9-ad7a5ee917aa None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:53.062790 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e70c76-5c12-4c44-9292-32167e84454c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:53.063526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e70c76-5c12-4c44-9292-32167e84454c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:53.063830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e70c76-5c12-4c44-9292-32167e84454c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:53.064134 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e70c76-5c12-4c44-9292-32167e84454c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:53.074704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e70c76-5c12-4c44-9292-32167e84454c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:53.084814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-22e70c76-5c12-4c44-9292-32167e84454c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:53.084814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:53.084814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:53.084814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:53.084814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:53.084814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:53.084814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:53.084814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:53.084814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:53.084814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:53.084814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:53.084814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:53.084814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:53.084814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:53.084814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:53.084814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:53.086871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:53.086871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:53.086871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:53.086871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:53.086871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:53.086871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:53.086871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:53.086871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:53.086871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:53.086871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:53.086871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:53.086871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:53.086871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:53.086871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:53.086871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:53.086871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:53.088849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:53.088849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:53.088849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:53.088849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:53.088849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:53.088849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:53.088849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:53.088849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:53.088849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:53.088849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:53.088849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:53.088849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:53.088849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 388/789] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:54:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:54:53.088849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2469044-e133-40a5-abac-1fd7075d2cb3 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:53.090692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2469044-e133-40a5-abac-1fd7075d2cb3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:53.090692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2469044-e133-40a5-abac-1fd7075d2cb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:53.105894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-a2469044-e133-40a5-abac-1fd7075d2cb3 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:53.107061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a2469044-e133-40a5-abac-1fd7075d2cb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:53.118116 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:53.126583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c0ff9-0222-4531-b9d0-0c5da3599958 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:53.128152 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:53.164543 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be761101-c882-4bb9-9107-515d79e9b24c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:53.165460 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be761101-c882-4bb9-9107-515d79e9b24c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:53.165752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be761101-c882-4bb9-9107-515d79e9b24c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:53.166035 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be761101-c882-4bb9-9107-515d79e9b24c None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:53.167187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be761101-c882-4bb9-9107-515d79e9b24c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:53.167754 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be761101-c882-4bb9-9107-515d79e9b24c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:53.169506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be761101-c882-4bb9-9107-515d79e9b24c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:53.182925 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 389/791] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:53 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:54:53.183103 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 402/791] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:53.184099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2469044-e133-40a5-abac-1fd7075d2cb3 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:53.189884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be761101-c882-4bb9-9107-515d79e9b24c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:53.310709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2469044-e133-40a5-abac-1fd7075d2cb3 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:53.329572 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be761101-c882-4bb9-9107-515d79e9b24c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:53.351718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422df604-2bd0-4147-8303-a1a93a7ca27b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:53.352458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422df604-2bd0-4147-8303-a1a93a7ca27b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:53.352751 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422df604-2bd0-4147-8303-a1a93a7ca27b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:53.353030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422df604-2bd0-4147-8303-a1a93a7ca27b None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:53.354307 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422df604-2bd0-4147-8303-a1a93a7ca27b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:53.354626 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422df604-2bd0-4147-8303-a1a93a7ca27b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:53.356443 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422df604-2bd0-4147-8303-a1a93a7ca27b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:53.368377 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 390/792] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:53.369224 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad536fd-f7bf-4e97-84de-da46f832dc2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:53.369371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422df604-2bd0-4147-8303-a1a93a7ca27b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:53.370550 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad536fd-f7bf-4e97-84de-da46f832dc2d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:53.371131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad536fd-f7bf-4e97-84de-da46f832dc2d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:53.371611 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad536fd-f7bf-4e97-84de-da46f832dc2d None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:53.380695 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad536fd-f7bf-4e97-84de-da46f832dc2d None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:53.382567 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad536fd-f7bf-4e97-84de-da46f832dc2d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:53.382710 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:53.400362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-dad536fd-f7bf-4e97-84de-da46f832dc2d None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 15:54:53.400362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:53.400362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:53.400362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:53.400362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:53.400362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:53.400362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:53.400362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:53.400362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:53.400362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:53.400362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:53.400362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:53.400362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:53.400362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:53.400362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:53.400362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:53.401727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:53.402789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:53.402789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:53.402789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:53.402789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:53.402789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:53.402789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:53.402789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:53.402789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:53.402789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 15:54:53.402789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:53.410082 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 403/793] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:54:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:54:53.411483 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad536fd-f7bf-4e97-84de-da46f832dc2d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:53.440758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:53.484377 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422df604-2bd0-4147-8303-a1a93a7ca27b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:53.528845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b14ca2-ac9e-4559-902c-0fd69e104190 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:53.530253 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b14ca2-ac9e-4559-902c-0fd69e104190 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:53.530660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b14ca2-ac9e-4559-902c-0fd69e104190 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:53.530949 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b14ca2-ac9e-4559-902c-0fd69e104190 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:53.535341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad536fd-f7bf-4e97-84de-da46f832dc2d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:53.544151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b14ca2-ac9e-4559-902c-0fd69e104190 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:53.555108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-d2b14ca2-ac9e-4559-902c-0fd69e104190 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:53.555108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:53.555108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:53.555108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:53.555108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:53.555108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:53.555108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:53.555108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:53.555108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:53.555108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:53.555108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:53.555108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:53.555108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:53.555108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:53.555108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:53.555108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:53.556348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:53.556348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:53.556348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:53.556348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:53.556348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:53.556348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:53.556348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:53.556348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:53.556348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:53.556348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:53.556348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:53.556348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:53.556348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:53.556348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:53.556348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:53.556348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:53.557383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:53.557383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:53.557383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:53.557383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:53.557383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:53.557383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:53.557383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:53.557383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:53.557383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:53.557383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:53.557383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:53.557383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:53.557383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 391/794] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:53 2022] GET /identity/v3/roles/service => generated 85 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:53.558276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b14ca2-ac9e-4559-902c-0fd69e104190 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:53.583774 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb15e1b-4df6-4b18-88c6-c034f52358ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:53.584861 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb15e1b-4df6-4b18-88c6-c034f52358ec None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:53.585298 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb15e1b-4df6-4b18-88c6-c034f52358ec None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:53.585727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb15e1b-4df6-4b18-88c6-c034f52358ec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:53.598042 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb15e1b-4df6-4b18-88c6-c034f52358ec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:53.599716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb15e1b-4df6-4b18-88c6-c034f52358ec None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:53.618342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-fcb15e1b-4df6-4b18-88c6-c034f52358ec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:53.618342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:53.618342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:53.618342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:53.618342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:53.618342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:53.618342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:53.618342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:53.618342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:53.618342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:53.618342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:53.618342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:53.618342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:53.618342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:53.618342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:53.619470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:53.619470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:53.619470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:53.619470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:53.619470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:53.619470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:53.619470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:53.619470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:53.619470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:53.619470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:53.619470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:53.619470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:53.619470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:53.619470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:53.619470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:53.619470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:53.619470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:53.620650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:53.620650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:53.620650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:53.620650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:53.622610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 404/795] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:53.623708 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb15e1b-4df6-4b18-88c6-c034f52358ec None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:53.642274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:53.684804 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b14ca2-ac9e-4559-902c-0fd69e104190 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:53.724950 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bb60f-5065-416d-971e-1d377c92f5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:53.725629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bb60f-5065-416d-971e-1d377c92f5a9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:53.725912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bb60f-5065-416d-971e-1d377c92f5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:53.726191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f99bb60f-5065-416d-971e-1d377c92f5a9 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:53.727031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99bb60f-5065-416d-971e-1d377c92f5a9 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:53.727344 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99bb60f-5065-416d-971e-1d377c92f5a9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:53.728817 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f99bb60f-5065-416d-971e-1d377c92f5a9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:53.729067 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:53.756430 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 392/796] 10.222.0.85 () {56 vars in 1135 bytes} [Wed May 25 15:54:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:53.756430 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bb60f-5065-416d-971e-1d377c92f5a9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:53.770929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb15e1b-4df6-4b18-88c6-c034f52358ec None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:53.816269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c667fc-b96e-46e9-b310-3609acaca573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:53.817805 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c667fc-b96e-46e9-b310-3609acaca573 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:53.818317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c667fc-b96e-46e9-b310-3609acaca573 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:53.819038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c667fc-b96e-46e9-b310-3609acaca573 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:53.820245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c667fc-b96e-46e9-b310-3609acaca573 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:53.820744 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c667fc-b96e-46e9-b310-3609acaca573 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:53.822912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c667fc-b96e-46e9-b310-3609acaca573 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:53.834789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 405/797] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:53 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:53.835945 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c667fc-b96e-46e9-b310-3609acaca573 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:53.883063 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f99bb60f-5065-416d-971e-1d377c92f5a9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:53.915672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:53.927538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca3510-18f5-4adf-bf28-784f1f281da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:53.928396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca3510-18f5-4adf-bf28-784f1f281da2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:53.928741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca3510-18f5-4adf-bf28-784f1f281da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:53.929026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca3510-18f5-4adf-bf28-784f1f281da2 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:53.930561 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca3510-18f5-4adf-bf28-784f1f281da2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:53.930877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca3510-18f5-4adf-bf28-784f1f281da2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:53.932318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca3510-18f5-4adf-bf28-784f1f281da2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:53.950990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 393/798] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:53.951901 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca3510-18f5-4adf-bf28-784f1f281da2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:53.969144 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c667fc-b96e-46e9-b310-3609acaca573 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:54.006627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad2d1c-2ee5-4a79-a154-1d1971f300c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:54.008059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad2d1c-2ee5-4a79-a154-1d1971f300c5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:54.008554 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad2d1c-2ee5-4a79-a154-1d1971f300c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:54.009072 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cad2d1c-2ee5-4a79-a154-1d1971f300c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:54.012435 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:54.023445 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad2d1c-2ee5-4a79-a154-1d1971f300c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:54.025419 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cad2d1c-2ee5-4a79-a154-1d1971f300c5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:54.048096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9cad2d1c-2ee5-4a79-a154-1d1971f300c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:54.048096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:54.048096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:54.048096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:54.048096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:54.048096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:54.048096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:54.048096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:54.048096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:54.048096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:54.048096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:54.048096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:54.048096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:54.048096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:54.048096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:54.049094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:54.049094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:54.049094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:54.049094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:54.049094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:54.049094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:54.049094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:54.049094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:54.049094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:54.049094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:54.049094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:54.049094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:54.049094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:54.049094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:54.049094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:54.049094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:54.049094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:54.050012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:54.050012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:54.050012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:54.050012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:54.050012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 406/799] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:54.050308 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad2d1c-2ee5-4a79-a154-1d1971f300c5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:54.072559 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca3510-18f5-4adf-bf28-784f1f281da2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:54.107430 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d84c3a-3bb5-487e-b438-f8ffe897501c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:54.108202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d84c3a-3bb5-487e-b438-f8ffe897501c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:54.108491 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d84c3a-3bb5-487e-b438-f8ffe897501c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:54.108780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d84c3a-3bb5-487e-b438-f8ffe897501c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:54.119903 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d84c3a-3bb5-487e-b438-f8ffe897501c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:54.121435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d84c3a-3bb5-487e-b438-f8ffe897501c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:54.141594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-67d84c3a-3bb5-487e-b438-f8ffe897501c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:54.141594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:54.141594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:54.141594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:54.141594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:54.141594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:54.141594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:54.141594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:54.141594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:54.141594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:54.141594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:54.141594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:54.141594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:54.141594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:54.141594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:54.142608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:54.142608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:54.142608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:54.142608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:54.142608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:54.142608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:54.142608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:54.142608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:54.142608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:54.142608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:54.142608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:54.142608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:54.142608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:54.142608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:54.142608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:54.142608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:54.142608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:54.143513 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:54.143513 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:54.143513 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:54.143513 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:54.143513 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 394/800] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:54.144735 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a521d23-1051-4e12-9c41-9518e650cf4a None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:54.145029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a521d23-1051-4e12-9c41-9518e650cf4a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:54.145311 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a521d23-1051-4e12-9c41-9518e650cf4a None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:54.146724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 395/801] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:54.147282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a521d23-1051-4e12-9c41-9518e650cf4a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:54.162849 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cad2d1c-2ee5-4a79-a154-1d1971f300c5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:54.192038 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:54.198591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a957e-87f5-4342-9a7c-f092a17c6ddf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:54.199460 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a957e-87f5-4342-9a7c-f092a17c6ddf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:54.199747 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a957e-87f5-4342-9a7c-f092a17c6ddf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:54.200377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572a957e-87f5-4342-9a7c-f092a17c6ddf None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:54.208247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a957e-87f5-4342-9a7c-f092a17c6ddf None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:54.209736 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-572a957e-87f5-4342-9a7c-f092a17c6ddf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:54.224589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-572a957e-87f5-4342-9a7c-f092a17c6ddf None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 15:54:54.224589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:54.224589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:54.224589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:54.224589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:54.224589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:54.224589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:54.224589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:54.224589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:54.224589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:54.224589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:54.224589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:54.224589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:54.224589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:54.224589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:54.224589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:54.225577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:54.226528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:54.226528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:54.226528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:54.226528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:54.226528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:54.226528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:54.226528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:54.226528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:54.226528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 15:54:54.226528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:54.226528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 407/802] 10.222.0.85 () {56 vars in 1093 bytes} [Wed May 25 15:54:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:54.226528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a957e-87f5-4342-9a7c-f092a17c6ddf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:54.264741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a521d23-1051-4e12-9c41-9518e650cf4a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:54.282776 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:54.303700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde6c63f-22f1-4fcc-bcc5-c9ba91f3fdfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:54.304527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6c63f-22f1-4fcc-bcc5-c9ba91f3fdfd None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:54.304751 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6c63f-22f1-4fcc-bcc5-c9ba91f3fdfd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:54.305013 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde6c63f-22f1-4fcc-bcc5-c9ba91f3fdfd None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:54.305764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6c63f-22f1-4fcc-bcc5-c9ba91f3fdfd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:54.306086 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6c63f-22f1-4fcc-bcc5-c9ba91f3fdfd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:54.307527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dde6c63f-22f1-4fcc-bcc5-c9ba91f3fdfd None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:54.320377 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 396/803] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:54.321749 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a4efd-5d63-4ef2-963d-d00d2018caad None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:54.322047 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a4efd-5d63-4ef2-963d-d00d2018caad None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:54.322335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a4efd-5d63-4ef2-963d-d00d2018caad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:54.341188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-530a4efd-5d63-4ef2-963d-d00d2018caad None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:54.342620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-530a4efd-5d63-4ef2-963d-d00d2018caad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:54.351763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-572a957e-87f5-4342-9a7c-f092a17c6ddf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:54.387329 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba91802-142b-4c0d-9ac5-465183428bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:54.388044 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba91802-142b-4c0d-9ac5-465183428bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:54.388345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba91802-142b-4c0d-9ac5-465183428bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:54.388630 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba91802-142b-4c0d-9ac5-465183428bc5 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:54.389369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba91802-142b-4c0d-9ac5-465183428bc5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:54.389689 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba91802-142b-4c0d-9ac5-465183428bc5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:54.391091 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dba91802-142b-4c0d-9ac5-465183428bc5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:54.403573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 408/804] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:54.404626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba91802-142b-4c0d-9ac5-465183428bc5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:54.419415 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 397/805] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:54 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:54:54.420664 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a4efd-5d63-4ef2-963d-d00d2018caad None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:54.449677 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:54.522548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba91802-142b-4c0d-9ac5-465183428bc5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:54.524648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a4efd-5d63-4ef2-963d-d00d2018caad None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:54.532191 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:54.556975 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c92fc-ceca-4eaa-9405-a8297fbb2fbe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:54.557655 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c92fc-ceca-4eaa-9405-a8297fbb2fbe None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:54.557942 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c92fc-ceca-4eaa-9405-a8297fbb2fbe None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:54.558221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-624c92fc-ceca-4eaa-9405-a8297fbb2fbe None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:54.558995 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca020dc0-158a-413d-9eb2-3f25a759d66d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:54.559107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c92fc-ceca-4eaa-9405-a8297fbb2fbe None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:54.559421 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c92fc-ceca-4eaa-9405-a8297fbb2fbe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:54.559792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca020dc0-158a-413d-9eb2-3f25a759d66d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:54.560112 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca020dc0-158a-413d-9eb2-3f25a759d66d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:54.560386 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca020dc0-158a-413d-9eb2-3f25a759d66d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:54.560912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-624c92fc-ceca-4eaa-9405-a8297fbb2fbe None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:54.567029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca020dc0-158a-413d-9eb2-3f25a759d66d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:54.568500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca020dc0-158a-413d-9eb2-3f25a759d66d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:54.577894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 409/806] 10.222.0.85 () {56 vars in 1133 bytes} [Wed May 25 15:54:54 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:54.578686 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c92fc-ceca-4eaa-9405-a8297fbb2fbe None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:54.582408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-ca020dc0-158a-413d-9eb2-3f25a759d66d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:54.582408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:54.582408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:54.582408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:54.582408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:54.582408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:54.582408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:54.582408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:54.582408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:54.582408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:54.582408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:54.582408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:54.582408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:54.582408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:54.582408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:54.583445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:54.583445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:54.583445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:54.583445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:54.583445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:54.583445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:54.583445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:54.583445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:54.583445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:54.583445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:54.583445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:54.583445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:54.583445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:54.583445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:54.583445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:54.583445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:54.583445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:54.584408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:54.584408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:54.584408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:54.584408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:54.584408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:54.584408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:54.584408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:54.584408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:54.584408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 398/807] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:54.584408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca020dc0-158a-413d-9eb2-3f25a759d66d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:54.669348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:54.680491 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-624c92fc-ceca-4eaa-9405-a8297fbb2fbe None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:54.683187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca020dc0-158a-413d-9eb2-3f25a759d66d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:54.714828 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3967b7-c037-47f6-9bb5-9606838e8153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:54.715649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3967b7-c037-47f6-9bb5-9606838e8153 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:54.715973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3967b7-c037-47f6-9bb5-9606838e8153 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:54.716227 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18f48e3-37ac-4d12-b2b8-bd35010018c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:54.716337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3967b7-c037-47f6-9bb5-9606838e8153 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:54.717049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f48e3-37ac-4d12-b2b8-bd35010018c8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:54.717338 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f48e3-37ac-4d12-b2b8-bd35010018c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:54.717619 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18f48e3-37ac-4d12-b2b8-bd35010018c8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:54.718278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f48e3-37ac-4d12-b2b8-bd35010018c8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:54.719661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18f48e3-37ac-4d12-b2b8-bd35010018c8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:54.723236 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3967b7-c037-47f6-9bb5-9606838e8153 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:54.724709 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa3967b7-c037-47f6-9bb5-9606838e8153 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:54.731699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-a18f48e3-37ac-4d12-b2b8-bd35010018c8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 15:54:54.731699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:54.731699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:54:54.731699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:54:54.731699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:54:54.731699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:54:54.731699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:54:54.731699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:54:54.731699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 15:54:54.731699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:54.731699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:54:54.731699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:54.731699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:54.731699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:54.731699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:54.731699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:54.732613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:54.732613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:54.732613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:54.732613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:54.732613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:54.732613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:54.732613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:54.732613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:54:54.732613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:54:54.732613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:54:54.732613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:54:54.732613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:54.732613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:54.732613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:54.732613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:54.732613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:54.733467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:54.733467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:54.733467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:54.733467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:54.733467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:54.733467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:54.733467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:54.733467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:54.733467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:54.733467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:54.733467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:54.733467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:54:54.733467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:54:54.733467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. May 25 15:54:54.733467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:54.733467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 399/808] 10.222.0.85 () {56 vars in 1129 bytes} [Wed May 25 15:54:54 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 149 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:54.734906 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744e132-839d-42cd-a169-ee25ca8e162a None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:54.734906 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744e132-839d-42cd-a169-ee25ca8e162a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:54.735057 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c744e132-839d-42cd-a169-ee25ca8e162a None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:54.736471 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 400/809] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:54.738384 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744e132-839d-42cd-a169-ee25ca8e162a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:54.739475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-aa3967b7-c037-47f6-9bb5-9606838e8153 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 15:54:54.739475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:54.739475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:54.739475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:54.739475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:54.739475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:54.739475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:54.739475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:54.739475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:54.739475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:54.739475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:54.739475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:54.739475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:54.739475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:54.739475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:54.739475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:54.742219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:54.743237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:54.743237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:54.743237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:54.743237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:54.743237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:54.743237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:54.743237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:54.743237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:54.743237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 15:54:54.743237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:54.743237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 410/810] 10.222.0.85 () {56 vars in 1117 bytes} [Wed May 25 15:54:54 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:54.743237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3967b7-c037-47f6-9bb5-9606838e8153 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:54.772101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:54.847794 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3967b7-c037-47f6-9bb5-9606838e8153 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:54.855418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c744e132-839d-42cd-a169-ee25ca8e162a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:54.881818 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b08d5-b237-4890-88af-ecf1642a8260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:54.882571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b08d5-b237-4890-88af-ecf1642a8260 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:54.882867 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b08d5-b237-4890-88af-ecf1642a8260 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:54.883150 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b08d5-b237-4890-88af-ecf1642a8260 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:54.883987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b08d5-b237-4890-88af-ecf1642a8260 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:54.884310 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b08d5-b237-4890-88af-ecf1642a8260 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:54.885771 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b08d5-b237-4890-88af-ecf1642a8260 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:54.888334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1104bd-9826-4524-bbdb-382ad5f93750 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:54.888986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1104bd-9826-4524-bbdb-382ad5f93750 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:54.889275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1104bd-9826-4524-bbdb-382ad5f93750 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:54.889558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c1104bd-9826-4524-bbdb-382ad5f93750 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:54.897317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 411/811] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:54.898112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b08d5-b237-4890-88af-ecf1642a8260 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:54.913925 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:54.960727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1104bd-9826-4524-bbdb-382ad5f93750 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:54.962133 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c1104bd-9826-4524-bbdb-382ad5f93750 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:54.983781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:55.000784 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b08d5-b237-4890-88af-ecf1642a8260 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:55.035809 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fa8d086-af29-4c67-8f7b-ff19907e7e61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:55.036517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8d086-af29-4c67-8f7b-ff19907e7e61 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:55.036808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8d086-af29-4c67-8f7b-ff19907e7e61 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:55.037092 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fa8d086-af29-4c67-8f7b-ff19907e7e61 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:55.037806 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8d086-af29-4c67-8f7b-ff19907e7e61 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:55.038122 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8d086-af29-4c67-8f7b-ff19907e7e61 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:55.039522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fa8d086-af29-4c67-8f7b-ff19907e7e61 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:55.044369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 412/812] 10.222.0.85 () {56 vars in 1124 bytes} [Wed May 25 15:54:54 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:55.045347 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0518ea-7c57-476c-8c4b-f6428bd1ed20 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:55.045642 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0518ea-7c57-476c-8c4b-f6428bd1ed20 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:55.045923 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd0518ea-7c57-476c-8c4b-f6428bd1ed20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:55.063681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dd0518ea-7c57-476c-8c4b-f6428bd1ed20 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:55.064903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dd0518ea-7c57-476c-8c4b-f6428bd1ed20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:55.066656 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 401/813] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:54 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:54:55.067521 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1104bd-9826-4524-bbdb-382ad5f93750 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:55.120510 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:55.132491 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 413/814] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:55 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:54:55.133327 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0518ea-7c57-476c-8c4b-f6428bd1ed20 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:55.169319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c1104bd-9826-4524-bbdb-382ad5f93750 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:55.203725 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1860d-f870-464e-8f04-c53e4d0387eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:55.204446 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1860d-f870-464e-8f04-c53e4d0387eb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:55.204731 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1860d-f870-464e-8f04-c53e4d0387eb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:55.205022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e1860d-f870-464e-8f04-c53e4d0387eb None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:55.205837 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1860d-f870-464e-8f04-c53e4d0387eb None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:55.206151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1860d-f870-464e-8f04-c53e4d0387eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:55.207964 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e1860d-f870-464e-8f04-c53e4d0387eb None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:55.213314 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:55.225956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 402/815] 10.222.0.85 () {56 vars in 1149 bytes} [Wed May 25 15:54:55 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:55.226842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1860d-f870-464e-8f04-c53e4d0387eb None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:55.233694 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd0518ea-7c57-476c-8c4b-f6428bd1ed20 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:55.266378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16776e21-c7c6-488c-8908-42e0582ccc8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:55.267285 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16776e21-c7c6-488c-8908-42e0582ccc8e None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:55.267577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16776e21-c7c6-488c-8908-42e0582ccc8e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:55.267883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16776e21-c7c6-488c-8908-42e0582ccc8e None admin] PATH_INFO: `/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/77fda27641c84fccb68a5cdaf4e8dc4b/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:55.269838 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16776e21-c7c6-488c-8908-42e0582ccc8e None admin] RBAC: Authorizing `identity:create_grant(project_id=5584ebdaf133444ca69efc5554bf48e9, user_id=77fda27641c84fccb68a5cdaf4e8dc4b, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:55.271359 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16776e21-c7c6-488c-8908-42e0582ccc8e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:55.284728 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 414/816] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:54:55 2022] PUT /identity/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/77fda27641c84fccb68a5cdaf4e8dc4b/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:54:55.285752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16776e21-c7c6-488c-8908-42e0582ccc8e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:55.336500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e1860d-f870-464e-8f04-c53e4d0387eb None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:55.349476 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:55.378038 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915df14-37f2-404a-8d35-37c5d802261f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:55.378730 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915df14-37f2-404a-8d35-37c5d802261f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:55.379018 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915df14-37f2-404a-8d35-37c5d802261f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:55.379297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9915df14-37f2-404a-8d35-37c5d802261f None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:55.380188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915df14-37f2-404a-8d35-37c5d802261f None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:55.380510 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915df14-37f2-404a-8d35-37c5d802261f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:55.381930 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9915df14-37f2-404a-8d35-37c5d802261f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:55.392683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 403/817] 10.222.0.85 () {56 vars in 1124 bytes} [Wed May 25 15:54:55 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:55.392683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915df14-37f2-404a-8d35-37c5d802261f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:55.416010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16776e21-c7c6-488c-8908-42e0582ccc8e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:55.455665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3881d-38a1-4dba-8527-52161270c2ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:55.456895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3881d-38a1-4dba-8527-52161270c2ee None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:55.457187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3881d-38a1-4dba-8527-52161270c2ee None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:55.457663 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3de3881d-38a1-4dba-8527-52161270c2ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:55.467965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:55.471115 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3881d-38a1-4dba-8527-52161270c2ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:55.472825 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3de3881d-38a1-4dba-8527-52161270c2ee None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:55.499075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3de3881d-38a1-4dba-8527-52161270c2ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:55.499075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:55.499075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:55.499075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:55.499075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:55.499075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:55.499075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:55.499075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:55.499075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:55.499075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:55.499075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:55.499075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:55.499075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:55.499075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:55.499075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:55.500141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:55.500141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:55.500141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:55.500141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:55.500141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:55.500141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:55.500141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:55.500141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:55.500141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:55.500141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:55.500141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:55.500141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:55.500141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:55.500141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:55.500141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:55.500141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:55.500141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:55.501281 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:55.501281 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:55.501281 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:55.501281 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:55.501281 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 415/818] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:55.501574 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3881d-38a1-4dba-8527-52161270c2ee None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:55.511214 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9915df14-37f2-404a-8d35-37c5d802261f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:55.547643 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039ad5c-2649-4a28-aa81-61d06f0c5b6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:55.548623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039ad5c-2649-4a28-aa81-61d06f0c5b6e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:55.548948 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039ad5c-2649-4a28-aa81-61d06f0c5b6e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:55.549269 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039ad5c-2649-4a28-aa81-61d06f0c5b6e None admin] PATH_INFO: `/v3/services/network` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:55.549927 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039ad5c-2649-4a28-aa81-61d06f0c5b6e None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:55.551374 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039ad5c-2649-4a28-aa81-61d06f0c5b6e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:55.563234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-5039ad5c-2649-4a28-aa81-61d06f0c5b6e None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. May 25 15:54:55.563234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:55.563234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:54:55.563234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:54:55.563234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:54:55.563234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:54:55.563234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:54:55.563234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:54:55.563234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 15:54:55.563234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:55.563234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:54:55.563234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:55.563234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:55.563234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:55.563234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:55.563234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:55.563234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:55.564294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:55.564294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:55.564294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:55.564294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:55.564294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:55.564294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:55.564294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:54:55.564294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:54:55.564294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:54:55.564294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:54:55.564294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:55.564294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:55.564294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:55.564294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:55.564294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:55.564294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:55.565140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:55.565140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:55.565140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:55.565140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:55.565140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:55.565140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:55.565140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:55.565140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:55.565140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:55.565140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:55.565140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:54:55.565140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:54:55.565140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. May 25 15:54:55.565140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:55.565140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 404/819] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:55 2022] GET /identity/v3/services/network => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:55.565943 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039ad5c-2649-4a28-aa81-61d06f0c5b6e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:55.609291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:55.609491 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3de3881d-38a1-4dba-8527-52161270c2ee None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:55.648078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5531df-d0da-409a-9a47-e213f765bca5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:55.648884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5531df-d0da-409a-9a47-e213f765bca5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:55.649172 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5531df-d0da-409a-9a47-e213f765bca5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:55.649451 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5531df-d0da-409a-9a47-e213f765bca5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:55.662232 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5531df-d0da-409a-9a47-e213f765bca5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:55.663751 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a5531df-d0da-409a-9a47-e213f765bca5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:55.676200 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039ad5c-2649-4a28-aa81-61d06f0c5b6e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:55.682108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8a5531df-d0da-409a-9a47-e213f765bca5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:55.682108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:55.682108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:55.682108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:55.682108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:55.682108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:55.682108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:55.682108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:55.682108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:55.682108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:55.682108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:55.682108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:55.682108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:55.682108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:55.682108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:55.683010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:55.683010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:55.683010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:55.683010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:55.683010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:55.683010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:55.683010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:55.683010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:55.683010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:55.683010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:55.683010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:55.683010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:55.683010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:55.683010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:55.683010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:55.683010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:55.683010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:55.688833 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:55.688833 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:55.688833 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:55.688833 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:55.688833 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 416/820] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:55.688833 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5531df-d0da-409a-9a47-e213f765bca5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:55.719133 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cd403-717f-4f74-8c51-0327d775e626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:55.719771 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cd403-717f-4f74-8c51-0327d775e626 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:55.720093 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cd403-717f-4f74-8c51-0327d775e626 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:55.720376 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d1cd403-717f-4f74-8c51-0327d775e626 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:55.721184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1cd403-717f-4f74-8c51-0327d775e626 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:55.722622 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d1cd403-717f-4f74-8c51-0327d775e626 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:55.734182 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:55.745961 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 405/821] 10.222.0.85 () {60 vars in 1144 bytes} [Wed May 25 15:54:55 2022] POST /identity/v3/endpoints => generated 346 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:54:55.747093 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cd403-717f-4f74-8c51-0327d775e626 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:55.811608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5531df-d0da-409a-9a47-e213f765bca5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:55.856727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22baec46-6cf9-4502-96a1-7da722f5ad2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:55.857603 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22baec46-6cf9-4502-96a1-7da722f5ad2d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:55.857983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22baec46-6cf9-4502-96a1-7da722f5ad2d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:55.858354 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22baec46-6cf9-4502-96a1-7da722f5ad2d None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:55.859204 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22baec46-6cf9-4502-96a1-7da722f5ad2d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:55.859573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22baec46-6cf9-4502-96a1-7da722f5ad2d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:55.861098 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22baec46-6cf9-4502-96a1-7da722f5ad2d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:55.878117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 417/822] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:55.878117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22baec46-6cf9-4502-96a1-7da722f5ad2d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:55.890212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:55.904388 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d1cd403-717f-4f74-8c51-0327d775e626 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:55.958266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700dc9d-4b92-4485-878e-cc3c316a1735 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:55.959298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700dc9d-4b92-4485-878e-cc3c316a1735 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:55.959795 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700dc9d-4b92-4485-878e-cc3c316a1735 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:55.960308 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700dc9d-4b92-4485-878e-cc3c316a1735 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:55.961956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700dc9d-4b92-4485-878e-cc3c316a1735 None admin] RBAC: Adding query filter params (name=network) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:55.962445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700dc9d-4b92-4485-878e-cc3c316a1735 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:55.964533 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1700dc9d-4b92-4485-878e-cc3c316a1735 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:55.971511 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 406/823] 10.222.0.85 () {56 vars in 1112 bytes} [Wed May 25 15:54:55 2022] GET /identity/v3/services?name=network => generated 124 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:55.972588 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700dc9d-4b92-4485-878e-cc3c316a1735 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:55.992820 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:55.997389 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22baec46-6cf9-4502-96a1-7da722f5ad2d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:56.043139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ffa91-7aaa-451a-984a-546aed97b628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:56.043868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ffa91-7aaa-451a-984a-546aed97b628 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:56.044147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ffa91-7aaa-451a-984a-546aed97b628 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:56.044433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-335ffa91-7aaa-451a-984a-546aed97b628 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:56.045735 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ffa91-7aaa-451a-984a-546aed97b628 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:56.046059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ffa91-7aaa-451a-984a-546aed97b628 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:56.048053 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-335ffa91-7aaa-451a-984a-546aed97b628 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:56.065486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 418/824] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:56.066436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ffa91-7aaa-451a-984a-546aed97b628 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:56.102402 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1700dc9d-4b92-4485-878e-cc3c316a1735 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:56.144765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de773408-aa38-4928-a0fa-f793273df657 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:56.145592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de773408-aa38-4928-a0fa-f793273df657 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:56.145880 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de773408-aa38-4928-a0fa-f793273df657 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:56.146160 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de773408-aa38-4928-a0fa-f793273df657 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:56.154572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de773408-aa38-4928-a0fa-f793273df657 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:56.156139 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de773408-aa38-4928-a0fa-f793273df657 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:56.171025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-de773408-aa38-4928-a0fa-f793273df657 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:56.171025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:56.171025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:56.171025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:56.171025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:56.171025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:56.171025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:56.171025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:56.171025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:56.171025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:56.171025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:56.171025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:56.171025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:56.171025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:56.171025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:56.171958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:56.171958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:56.171958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:56.171958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:56.171958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:56.171958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:56.171958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:56.171958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:56.171958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:56.171958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:56.171958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:56.171958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:56.171958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:56.171958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:56.171958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:56.171958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:56.171958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:56.172859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:56.172859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:56.172859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:56.172859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:56.172859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:56.172859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:56.172859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:56.172859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:56.172859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 407/825] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:56.173823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de773408-aa38-4928-a0fa-f793273df657 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:56.178617 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-335ffa91-7aaa-451a-984a-546aed97b628 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:56.192855 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:56.215209 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b37cc-0373-406f-a779-12347159ba2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:56.215927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b37cc-0373-406f-a779-12347159ba2f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:56.216215 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b37cc-0373-406f-a779-12347159ba2f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:56.216496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55b37cc-0373-406f-a779-12347159ba2f None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:56.217268 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b37cc-0373-406f-a779-12347159ba2f None admin] RBAC: Adding query filter params (type=network) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:56.217573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b37cc-0373-406f-a779-12347159ba2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:56.219019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55b37cc-0373-406f-a779-12347159ba2f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:56.229952 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 419/826] 10.222.0.85 () {56 vars in 1112 bytes} [Wed May 25 15:54:56 2022] GET /identity/v3/services?type=network => generated 350 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:56.229952 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b37cc-0373-406f-a779-12347159ba2f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:56.252552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:56.285104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de773408-aa38-4928-a0fa-f793273df657 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:56.321829 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affc5dc-02bc-4f20-8b16-0ed91cfae37a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:56.322887 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affc5dc-02bc-4f20-8b16-0ed91cfae37a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:56.323328 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affc5dc-02bc-4f20-8b16-0ed91cfae37a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:56.323769 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0affc5dc-02bc-4f20-8b16-0ed91cfae37a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:56.331170 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affc5dc-02bc-4f20-8b16-0ed91cfae37a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:56.333192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0affc5dc-02bc-4f20-8b16-0ed91cfae37a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:56.334836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55b37cc-0373-406f-a779-12347159ba2f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:56.347551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-0affc5dc-02bc-4f20-8b16-0ed91cfae37a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:56.347551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:56.347551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:56.347551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:56.347551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:56.347551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:56.347551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:56.347551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:56.347551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:56.347551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:56.347551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:56.347551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:56.347551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:54:56.347551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:54:56.347551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:54:56.349002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:54:56.349002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:56.349002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:56.349002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:56.349002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:56.349002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:56.349002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:56.349002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:56.349002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:56.349002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:56.349002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:56.349002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:56.349002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:56.349002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:56.349002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:56.349002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:56.349002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:56.350023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:54:56.350023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:54:56.350023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:54:56.350023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:54:56.350023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:54:56.350023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:54:56.350023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:54:56.350023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:56.352120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 408/827] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:54:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:56.353112 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affc5dc-02bc-4f20-8b16-0ed91cfae37a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:56.380916 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307daa5d-131f-443e-b09e-20debde68118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:56.382243 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307daa5d-131f-443e-b09e-20debde68118 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:56.382682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307daa5d-131f-443e-b09e-20debde68118 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:56.383106 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-307daa5d-131f-443e-b09e-20debde68118 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:56.384090 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307daa5d-131f-443e-b09e-20debde68118 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:56.385082 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307daa5d-131f-443e-b09e-20debde68118 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:56.386998 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-307daa5d-131f-443e-b09e-20debde68118 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:56.404145 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 420/828] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:56.405804 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307daa5d-131f-443e-b09e-20debde68118 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:56.458052 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:56.486342 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0affc5dc-02bc-4f20-8b16-0ed91cfae37a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:56.528594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:56.532601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9785aac-6a58-4d01-8877-67fb67a9eef3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:56.533988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9785aac-6a58-4d01-8877-67fb67a9eef3 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:56.534368 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9785aac-6a58-4d01-8877-67fb67a9eef3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:56.534876 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9785aac-6a58-4d01-8877-67fb67a9eef3 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:56.537324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9785aac-6a58-4d01-8877-67fb67a9eef3 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:56.539571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9785aac-6a58-4d01-8877-67fb67a9eef3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:56.543249 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-307daa5d-131f-443e-b09e-20debde68118 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:56.567480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 409/829] 10.222.0.85 () {60 vars in 1144 bytes} [Wed May 25 15:54:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:54:56.568660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9785aac-6a58-4d01-8877-67fb67a9eef3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:56.616725 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd744a-6687-4241-99e3-ece2f2aad12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:56.617550 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd744a-6687-4241-99e3-ece2f2aad12e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:56.617910 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd744a-6687-4241-99e3-ece2f2aad12e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:56.618264 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bd744a-6687-4241-99e3-ece2f2aad12e None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:56.619135 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd744a-6687-4241-99e3-ece2f2aad12e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:56.619526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd744a-6687-4241-99e3-ece2f2aad12e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:56.621331 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bd744a-6687-4241-99e3-ece2f2aad12e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:56.647802 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 421/830] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:54:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:56.656159 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd744a-6687-4241-99e3-ece2f2aad12e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:56.697110 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9785aac-6a58-4d01-8877-67fb67a9eef3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:56.738679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b044b7ff-9947-4292-8024-056e7c718566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:56.739735 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044b7ff-9947-4292-8024-056e7c718566 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:56.740185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044b7ff-9947-4292-8024-056e7c718566 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:56.740606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b044b7ff-9947-4292-8024-056e7c718566 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:56.741650 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044b7ff-9947-4292-8024-056e7c718566 None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=4751abc9f67947b3ad17ba2b78ce2c12) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:56.742108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044b7ff-9947-4292-8024-056e7c718566 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:56.744305 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b044b7ff-9947-4292-8024-056e7c718566 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:56.746611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:56.753634 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 410/831] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:54:56 2022] GET /identity/v3/role_assignments?user.id=4751abc9f67947b3ad17ba2b78ce2c12&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:56.827092 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bd744a-6687-4241-99e3-ece2f2aad12e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:56.867771 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ec2643-0081-4a60-9c69-f2668573071f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:56.868730 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec2643-0081-4a60-9c69-f2668573071f None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:56.869021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec2643-0081-4a60-9c69-f2668573071f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:56.869309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ec2643-0081-4a60-9c69-f2668573071f None admin] PATH_INFO: `/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/066de1f11f724eee9df0377beb58d92c/roles/83bf7d9c29d4464dab8030f6d706cd6d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:56.883024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:56.888040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec2643-0081-4a60-9c69-f2668573071f None admin] RBAC: Authorizing `identity:create_grant(project_id=5584ebdaf133444ca69efc5554bf48e9, user_id=066de1f11f724eee9df0377beb58d92c, role_id=83bf7d9c29d4464dab8030f6d706cd6d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:56.889661 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ec2643-0081-4a60-9c69-f2668573071f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:56.912260 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 422/832] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:54:56 2022] PUT /identity/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/066de1f11f724eee9df0377beb58d92c/roles/83bf7d9c29d4464dab8030f6d706cd6d => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:54:57.585349 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1224e-d641-4f2c-950a-3cb1e74214a8 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:57.585855 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1224e-d641-4f2c-950a-3cb1e74214a8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:57.585855 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a1224e-d641-4f2c-950a-3cb1e74214a8 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:57.587486 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 411/833] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:57.593054 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f73c9-b068-444f-9773-dcf68a0bc77e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:57.593492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f73c9-b068-444f-9773-dcf68a0bc77e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:57.593686 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-834f73c9-b068-444f-9773-dcf68a0bc77e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:57.610895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-834f73c9-b068-444f-9773-dcf68a0bc77e None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:57.612125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-834f73c9-b068-444f-9773-dcf68a0bc77e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:57.689869 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 423/834] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:57 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:54:57.715961 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1224e-d641-4f2c-950a-3cb1e74214a8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:57.828976 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a1224e-d641-4f2c-950a-3cb1e74214a8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:57.880061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70aa3536-f5ac-493d-a05b-b8dc681f94cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:57.880937 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa3536-f5ac-493d-a05b-b8dc681f94cc None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:57.881234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa3536-f5ac-493d-a05b-b8dc681f94cc None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:57.881521 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70aa3536-f5ac-493d-a05b-b8dc681f94cc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:57.904676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70aa3536-f5ac-493d-a05b-b8dc681f94cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:57.919444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-70aa3536-f5ac-493d-a05b-b8dc681f94cc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:57.919444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:57.919444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:57.919444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:57.919444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:57.919444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:57.919444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:57.919444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:57.919444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:57.919444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:57.919444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:57.919444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:57.919444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:57.919444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:57.919444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:57.919444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:57.920441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:57.920441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:57.920441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:57.920441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:57.920441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:57.920441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:57.920441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:57.920441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:57.920441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:57.920441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:57.920441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:57.920441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:57.920441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:57.920441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:57.920441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:57.920441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:57.921295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:57.921295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:57.921295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:57.921295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:57.921295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:57.921295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:57.921295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:57.921295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:57.921295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:57.921295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:57.921295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:54:57.921295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:57.921295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 412/835] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:54:57 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:57.945316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f73c9-b068-444f-9773-dcf68a0bc77e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:58.158039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-834f73c9-b068-444f-9773-dcf68a0bc77e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:58.211064 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:58.231743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96395d9e-cb3c-479e-8276-057725a12abd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:58.232460 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96395d9e-cb3c-479e-8276-057725a12abd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:58.233469 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96395d9e-cb3c-479e-8276-057725a12abd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:58.233469 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96395d9e-cb3c-479e-8276-057725a12abd None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:58.233780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96395d9e-cb3c-479e-8276-057725a12abd None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:58.236156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96395d9e-cb3c-479e-8276-057725a12abd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:58.236156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96395d9e-cb3c-479e-8276-057725a12abd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:58.251976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 424/836] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:54:57 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:58.256508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f3c88-7e4f-493e-a0bf-1280181bd679 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:58.256811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f3c88-7e4f-493e-a0bf-1280181bd679 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:58.257098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234f3c88-7e4f-493e-a0bf-1280181bd679 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:58.259100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 413/837] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:58.260252 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96395d9e-cb3c-479e-8276-057725a12abd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:58.264189 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ad9b4-35ef-42ad-8585-d5fef9e7f5a2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:58.264718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ad9b4-35ef-42ad-8585-d5fef9e7f5a2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:58.265029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ad9b4-35ef-42ad-8585-d5fef9e7f5a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:58.294493 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-113ad9b4-35ef-42ad-8585-d5fef9e7f5a2 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:58.294493 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-113ad9b4-35ef-42ad-8585-d5fef9e7f5a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:58.377337 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:58.393736 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96395d9e-cb3c-479e-8276-057725a12abd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:58.395904 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 414/838] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:54:58.417384 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ad9b4-35ef-42ad-8585-d5fef9e7f5a2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:58.439803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877ca6a-ab0d-49c8-be10-861ea2175471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:58.440987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877ca6a-ab0d-49c8-be10-861ea2175471 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:58.441572 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877ca6a-ab0d-49c8-be10-861ea2175471 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:58.442065 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f877ca6a-ab0d-49c8-be10-861ea2175471 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:58.454415 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877ca6a-ab0d-49c8-be10-861ea2175471 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:58.456522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f877ca6a-ab0d-49c8-be10-861ea2175471 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:58.474816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f877ca6a-ab0d-49c8-be10-861ea2175471 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:58.474816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:58.474816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:58.474816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:58.474816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:58.474816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:58.474816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:58.474816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:58.474816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:58.474816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:58.474816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:58.474816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:58.474816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:58.474816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:58.474816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:58.475810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:58.475810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:58.475810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:58.475810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:58.475810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:58.475810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:58.475810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:58.475810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:58.475810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:58.475810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:58.475810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:58.475810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:58.475810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:58.475810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:58.475810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:58.475810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:58.475810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:58.476758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:58.476758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:58.476758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:58.476758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:58.479066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 425/839] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:58.484762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877ca6a-ab0d-49c8-be10-861ea2175471 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:58.522902 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:58.523234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ad9b4-35ef-42ad-8585-d5fef9e7f5a2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:58.560369 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5f34d-bdd7-4d8c-a9d7-1ce6979a56e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:58.561189 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5f34d-bdd7-4d8c-a9d7-1ce6979a56e0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:58.561478 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5f34d-bdd7-4d8c-a9d7-1ce6979a56e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:58.561756 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c5f34d-bdd7-4d8c-a9d7-1ce6979a56e0 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:58.562418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c5f34d-bdd7-4d8c-a9d7-1ce6979a56e0 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:58.564357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c5f34d-bdd7-4d8c-a9d7-1ce6979a56e0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:58.576593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-c1c5f34d-bdd7-4d8c-a9d7-1ce6979a56e0 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 15:54:58.576593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:58.576593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:54:58.576593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:54:58.576593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:54:58.576593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:54:58.576593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:54:58.576593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:54:58.576593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 15:54:58.576593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:58.576593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:54:58.576593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:58.576593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:58.576593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:58.576593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:58.576593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:58.576593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:58.577639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:58.577639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:58.577639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:58.577639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:58.577639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:58.577639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:58.577639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:54:58.577639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:54:58.577639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:54:58.577639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:54:58.577639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:58.577639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:58.577639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:58.577639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:58.577639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:58.577639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:58.578520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:58.578520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:58.578520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:58.578520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:58.578520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:58.578520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:58.578520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:58.578520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:58.578520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:58.578520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:58.578520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:54:58.578520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:54:58.578520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 15:54:58.578520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:58.583509 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 415/840] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:54:58 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:58.584677 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5f34d-bdd7-4d8c-a9d7-1ce6979a56e0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:58.601029 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f877ca6a-ab0d-49c8-be10-861ea2175471 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:58.638224 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31106e6-d676-46ce-a1e2-ceb81be1bcb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:58.638970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31106e6-d676-46ce-a1e2-ceb81be1bcb9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:58.639269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31106e6-d676-46ce-a1e2-ceb81be1bcb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:58.639603 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31106e6-d676-46ce-a1e2-ceb81be1bcb9 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:58.640394 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31106e6-d676-46ce-a1e2-ceb81be1bcb9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:58.640717 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31106e6-d676-46ce-a1e2-ceb81be1bcb9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:58.641685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:58.643849 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31106e6-d676-46ce-a1e2-ceb81be1bcb9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:58.662680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 426/841] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:58.668343 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31106e6-d676-46ce-a1e2-ceb81be1bcb9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:58.693346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c5f34d-bdd7-4d8c-a9d7-1ce6979a56e0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:58.729535 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7bcd2-e318-4ff2-8879-a25017f2a229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:58.730666 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7bcd2-e318-4ff2-8879-a25017f2a229 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:58.731174 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7bcd2-e318-4ff2-8879-a25017f2a229 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:58.731669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7bcd2-e318-4ff2-8879-a25017f2a229 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:58.732598 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7bcd2-e318-4ff2-8879-a25017f2a229 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:58.733112 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7bcd2-e318-4ff2-8879-a25017f2a229 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:58.734677 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7bcd2-e318-4ff2-8879-a25017f2a229 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:58.740275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 416/842] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:54:58 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:58.755845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7bcd2-e318-4ff2-8879-a25017f2a229 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:58.783422 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:58.790250 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31106e6-d676-46ce-a1e2-ceb81be1bcb9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:58.826888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704dd93-adc5-4406-a20c-7abf0ab34513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:58.828158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704dd93-adc5-4406-a20c-7abf0ab34513 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:58.828549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704dd93-adc5-4406-a20c-7abf0ab34513 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:58.828927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9704dd93-adc5-4406-a20c-7abf0ab34513 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:58.838770 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704dd93-adc5-4406-a20c-7abf0ab34513 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:58.841034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9704dd93-adc5-4406-a20c-7abf0ab34513 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:58.860893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9704dd93-adc5-4406-a20c-7abf0ab34513 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. May 25 15:54:58.860893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:58.860893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:58.860893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:58.860893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:58.860893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:58.860893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:58.860893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:58.860893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:58.860893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:58.860893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:58.860893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:58.860893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:54:58.860893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:54:58.860893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:54:58.860893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:58.862095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:58.863282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:58.863282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:58.863282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:54:58.863282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:54:58.863282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:54:58.863282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:54:58.863282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:54:58.863282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:54:58.863282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. May 25 15:54:58.863282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:58.867858 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 427/843] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:54:58 2022] GET /identity/v3/users/swift => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:54:58.868038 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7bcd2-e318-4ff2-8879-a25017f2a229 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:58.872209 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704dd93-adc5-4406-a20c-7abf0ab34513 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:58.887295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:58.907215 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459c691-9edf-4f06-ac3b-56e0f4f388e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:58.907921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459c691-9edf-4f06-ac3b-56e0f4f388e3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:58.908212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459c691-9edf-4f06-ac3b-56e0f4f388e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:58.908496 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4459c691-9edf-4f06-ac3b-56e0f4f388e3 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:58.909215 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459c691-9edf-4f06-ac3b-56e0f4f388e3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:58.909531 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459c691-9edf-4f06-ac3b-56e0f4f388e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:58.911036 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4459c691-9edf-4f06-ac3b-56e0f4f388e3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:58.924521 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 417/844] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:54:58 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:59.001856 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9704dd93-adc5-4406-a20c-7abf0ab34513 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:59.047437 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0931d69c-547f-4bc0-811d-8777dd0df3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:59.048627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931d69c-547f-4bc0-811d-8777dd0df3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:59.049126 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931d69c-547f-4bc0-811d-8777dd0df3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:59.049638 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0931d69c-547f-4bc0-811d-8777dd0df3b6 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:59.050707 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931d69c-547f-4bc0-811d-8777dd0df3b6 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:59.051259 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931d69c-547f-4bc0-811d-8777dd0df3b6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:59.053090 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0931d69c-547f-4bc0-811d-8777dd0df3b6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:59.075405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 428/845] 10.222.0.85 () {56 vars in 1135 bytes} [Wed May 25 15:54:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:54:59.081598 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459c691-9edf-4f06-ac3b-56e0f4f388e3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:59.188501 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808f99f-f593-4f20-9942-a371563d6353 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:59.188816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808f99f-f593-4f20-9942-a371563d6353 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:59.188997 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8808f99f-f593-4f20-9942-a371563d6353 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:59.190620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 429/846] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:59.195741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155a8e2-72f4-4cd1-84c7-40195f038477 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:59.196064 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155a8e2-72f4-4cd1-84c7-40195f038477 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:59.196356 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6155a8e2-72f4-4cd1-84c7-40195f038477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:59.204669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4459c691-9edf-4f06-ac3b-56e0f4f388e3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:59.213647 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6155a8e2-72f4-4cd1-84c7-40195f038477 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:59.214879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6155a8e2-72f4-4cd1-84c7-40195f038477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:59.223253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:59.250814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e167b7c-5976-42ae-af14-ff4ddddf746c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:59.252432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e167b7c-5976-42ae-af14-ff4ddddf746c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:59.252905 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e167b7c-5976-42ae-af14-ff4ddddf746c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:59.253191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e167b7c-5976-42ae-af14-ff4ddddf746c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:59.299299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 430/847] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:54:59.301043 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95d46d-2a6f-412e-8e86-ebe0f90d9a33 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:59.301562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95d46d-2a6f-412e-8e86-ebe0f90d9a33 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:59.302062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa95d46d-2a6f-412e-8e86-ebe0f90d9a33 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:59.303578 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 431/848] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:54:59.309651 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce23ade-b932-449c-b7d3-2c031ac67d6a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:59.310341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce23ade-b932-449c-b7d3-2c031ac67d6a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:59.310817 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ce23ade-b932-449c-b7d3-2c031ac67d6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:59.319368 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:59.329193 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1ce23ade-b932-449c-b7d3-2c031ac67d6a None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:54:59.330436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1ce23ade-b932-449c-b7d3-2c031ac67d6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:54:59.344474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e167b7c-5976-42ae-af14-ff4ddddf746c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:59.346478 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e167b7c-5976-42ae-af14-ff4ddddf746c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:59.400380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 432/849] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:54:59 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:54:59.401572 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce23ade-b932-449c-b7d3-2c031ac67d6a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:59.457891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 418/850] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:54:59 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:54:59.458591 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e167b7c-5976-42ae-af14-ff4ddddf746c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:59.487464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:59.521332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ce23ade-b932-449c-b7d3-2c031ac67d6a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:59.548064 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:59.560345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d5023-6a86-4760-8248-bee239697b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:59.561262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d5023-6a86-4760-8248-bee239697b30 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:59.561558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d5023-6a86-4760-8248-bee239697b30 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:59.561843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778d5023-6a86-4760-8248-bee239697b30 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:59.562514 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d5023-6a86-4760-8248-bee239697b30 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:59.563934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778d5023-6a86-4760-8248-bee239697b30 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:59.570966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e167b7c-5976-42ae-af14-ff4ddddf746c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:59.578348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-778d5023-6a86-4760-8248-bee239697b30 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 15:54:59.578348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:59.578348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:54:59.578348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:54:59.578348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:54:59.578348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:54:59.578348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:54:59.578348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:54:59.578348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 15:54:59.578348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:59.578348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:54:59.578348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:59.578348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:59.578348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:59.578348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:59.578348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:59.579443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:59.579443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:59.579443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:59.579443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:59.579443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:59.579443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:59.579443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:59.579443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:54:59.579443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:54:59.579443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:54:59.579443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:54:59.579443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:59.579443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:59.579443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:59.579443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:59.579443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:59.580420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:59.580420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:59.580420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:59.580420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:59.580420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:59.580420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:59.580420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:59.580420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:59.580420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:59.580420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:59.580420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:59.580420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:54:59.580420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:54:59.580420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 15:54:59.580420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:59.580420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 433/851] 10.222.0.85 () {56 vars in 1132 bytes} [Wed May 25 15:54:59 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:59.581717 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d5023-6a86-4760-8248-bee239697b30 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:59.614421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929a2db-4b43-4e31-a3ce-f671da0f55ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:59.615234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929a2db-4b43-4e31-a3ce-f671da0f55ca None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:59.615523 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929a2db-4b43-4e31-a3ce-f671da0f55ca None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:59.615812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c929a2db-4b43-4e31-a3ce-f671da0f55ca None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:59.626032 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c929a2db-4b43-4e31-a3ce-f671da0f55ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:54:59.633915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-c929a2db-4b43-4e31-a3ce-f671da0f55ca None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:59.633915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:59.633915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:59.633915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:59.633915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:59.633915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:59.633915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:59.633915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:59.633915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:59.633915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:59.633915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:59.633915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:59.633915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:54:59.633915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:54:59.633915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:54:59.633915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:54:59.634840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:54:59.634840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:54:59.634840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:54:59.634840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:54:59.634840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:59.634840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:59.634840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:59.634840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:59.634840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:59.634840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:59.634840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:59.634840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:54:59.634840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:59.634840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:59.634840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:59.634840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:59.635684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:59.635684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:59.635684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:59.635684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:59.635684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:54:59.635684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:54:59.635684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:54:59.635684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:54:59.635684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:54:59.635684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:54:59.635684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:54:59.635684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:54:59.635684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 419/852] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:54:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:59.637078 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929a2db-4b43-4e31-a3ce-f671da0f55ca None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:59.696310 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778d5023-6a86-4760-8248-bee239697b30 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:59.731580 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef55b9d-7eb4-44db-849b-71c22f9eaa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:59.732367 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef55b9d-7eb4-44db-849b-71c22f9eaa90 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:59.732663 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef55b9d-7eb4-44db-849b-71c22f9eaa90 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:59.732947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef55b9d-7eb4-44db-849b-71c22f9eaa90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:59.736231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:59.746503 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef55b9d-7eb4-44db-849b-71c22f9eaa90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:59.746720 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c929a2db-4b43-4e31-a3ce-f671da0f55ca None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:59.749889 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef55b9d-7eb4-44db-849b-71c22f9eaa90 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:59.769423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-fef55b9d-7eb4-44db-849b-71c22f9eaa90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:59.769423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:54:59.769423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:54:59.769423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:54:59.769423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:54:59.769423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:54:59.769423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:54:59.769423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:54:59.769423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:54:59.769423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:54:59.769423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:54:59.769423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:54:59.769423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:54:59.769423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:54:59.769423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:54:59.770387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:54:59.770387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:54:59.770387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:54:59.770387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:54:59.770387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:54:59.770387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:54:59.770387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:54:59.770387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:54:59.770387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:54:59.770387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:54:59.770387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:54:59.770387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:54:59.770387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:54:59.770387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:54:59.770387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:54:59.770387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:54:59.770387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:54:59.771328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:54:59.771328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:54:59.771328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:54:59.771328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:54:59.771328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 434/853] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:54:59.772258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef55b9d-7eb4-44db-849b-71c22f9eaa90 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:59.785333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034217c1-5142-4b17-8bc9-e84ea662b70d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:59.786375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034217c1-5142-4b17-8bc9-e84ea662b70d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:59.786671 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034217c1-5142-4b17-8bc9-e84ea662b70d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:59.786957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034217c1-5142-4b17-8bc9-e84ea662b70d None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:59.788025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034217c1-5142-4b17-8bc9-e84ea662b70d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:59.788611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034217c1-5142-4b17-8bc9-e84ea662b70d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:59.790174 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034217c1-5142-4b17-8bc9-e84ea662b70d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:59.800280 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 420/854] 10.222.0.85 () {56 vars in 1126 bytes} [Wed May 25 15:54:59 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:59.800280 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034217c1-5142-4b17-8bc9-e84ea662b70d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:59.802831 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:54:59.881345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef55b9d-7eb4-44db-849b-71c22f9eaa90 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:59.912719 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034217c1-5142-4b17-8bc9-e84ea662b70d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:54:59.916712 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992974a4-f0ff-4332-9d73-856e437d9570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:59.917426 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992974a4-f0ff-4332-9d73-856e437d9570 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:59.917767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992974a4-f0ff-4332-9d73-856e437d9570 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:59.918045 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-992974a4-f0ff-4332-9d73-856e437d9570 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:59.918823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992974a4-f0ff-4332-9d73-856e437d9570 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:59.919140 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992974a4-f0ff-4332-9d73-856e437d9570 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:59.920657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-992974a4-f0ff-4332-9d73-856e437d9570 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:59.929616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 435/855] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:54:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:59.935259 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992974a4-f0ff-4332-9d73-856e437d9570 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:59.950291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc74cc-4f4e-46a3-a3f0-ccff94d65fbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:54:59.950967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc74cc-4f4e-46a3-a3f0-ccff94d65fbb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:54:59.951258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc74cc-4f4e-46a3-a3f0-ccff94d65fbb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:54:59.951531 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bc74cc-4f4e-46a3-a3f0-ccff94d65fbb None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:54:59.952293 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc74cc-4f4e-46a3-a3f0-ccff94d65fbb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:54:59.952610 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc74cc-4f4e-46a3-a3f0-ccff94d65fbb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:54:59.953995 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45bc74cc-4f4e-46a3-a3f0-ccff94d65fbb None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:54:59.966587 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 421/856] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:54:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:54:59.967372 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc74cc-4f4e-46a3-a3f0-ccff94d65fbb None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:54:59.993285 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:00.028882 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:00.050519 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-992974a4-f0ff-4332-9d73-856e437d9570 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:00.085657 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bc74cc-4f4e-46a3-a3f0-ccff94d65fbb None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:00.086751 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77c13c-1355-437b-9ec7-400582c459c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:00.087561 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77c13c-1355-437b-9ec7-400582c459c3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:00.087904 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77c13c-1355-437b-9ec7-400582c459c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:00.088304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d77c13c-1355-437b-9ec7-400582c459c3 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:00.089009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77c13c-1355-437b-9ec7-400582c459c3 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:00.089383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77c13c-1355-437b-9ec7-400582c459c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:00.090839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d77c13c-1355-437b-9ec7-400582c459c3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:00.107922 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 436/857] 10.222.0.85 () {56 vars in 1126 bytes} [Wed May 25 15:54:59 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:00.107922 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77c13c-1355-437b-9ec7-400582c459c3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:00.140151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac4041a-c776-451c-a5d1-ebc72e4f32b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:00.141090 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4041a-c776-451c-a5d1-ebc72e4f32b2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:00.141703 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4041a-c776-451c-a5d1-ebc72e4f32b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:00.142052 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac4041a-c776-451c-a5d1-ebc72e4f32b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:00.154484 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac4041a-c776-451c-a5d1-ebc72e4f32b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:00.156001 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac4041a-c776-451c-a5d1-ebc72e4f32b2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:00.179250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-fac4041a-c776-451c-a5d1-ebc72e4f32b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:00.179250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:00.179250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:00.179250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:00.179250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:00.179250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:00.179250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:00.179250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:00.179250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:00.179250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:00.179250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:00.179250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:00.179250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:55:00.179250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:55:00.179250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:55:00.180263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:55:00.180263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:00.180263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:00.180263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:00.180263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:00.180263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:00.180263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:00.180263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:00.180263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:00.180263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:00.180263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:00.180263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:00.180263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:00.180263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:00.180263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:00.180263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:00.180263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:00.181166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:55:00.181166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:55:00.181166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:00.181166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:00.181560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 422/858] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:54:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:00.187165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac4041a-c776-451c-a5d1-ebc72e4f32b2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:00.228023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d77c13c-1355-437b-9ec7-400582c459c3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:00.244212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:00.267204 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebeaf9-1a7f-4677-8cf1-cd818052c31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:00.268073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebeaf9-1a7f-4677-8cf1-cd818052c31e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:00.268366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebeaf9-1a7f-4677-8cf1-cd818052c31e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:00.268645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ebeaf9-1a7f-4677-8cf1-cd818052c31e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:00.276611 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ebeaf9-1a7f-4677-8cf1-cd818052c31e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:00.278286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ebeaf9-1a7f-4677-8cf1-cd818052c31e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:00.299564 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac4041a-c776-451c-a5d1-ebc72e4f32b2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:00.304492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-38ebeaf9-1a7f-4677-8cf1-cd818052c31e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:00.304492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:00.304492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:00.304492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:00.304492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:00.304492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:00.304492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:00.304492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:00.304492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:00.304492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:00.304492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:00.304492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:00.304492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:55:00.304492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:55:00.304492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:55:00.305391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:55:00.305391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:00.305391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:00.305391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:00.305391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:00.305391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:00.305391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:00.305391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:00.305391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:00.305391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:00.305391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:00.305391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:00.305391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:00.305391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:00.305391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:00.305391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:00.305391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:00.306341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:55:00.306341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:55:00.306341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:55:00.306341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:55:00.306341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:55:00.306341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:55:00.306341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:00.306341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:00.310909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 437/859] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:55:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:00.312372 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebeaf9-1a7f-4677-8cf1-cd818052c31e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:00.332812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:00.348811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9c98f-c6ec-45ab-b7cd-b87daa61ce8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:00.349324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9c98f-c6ec-45ab-b7cd-b87daa61ce8e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:00.349619 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9c98f-c6ec-45ab-b7cd-b87daa61ce8e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:00.350403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde9c98f-c6ec-45ab-b7cd-b87daa61ce8e None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:00.350772 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9c98f-c6ec-45ab-b7cd-b87daa61ce8e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:00.351109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9c98f-c6ec-45ab-b7cd-b87daa61ce8e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:00.352724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fde9c98f-c6ec-45ab-b7cd-b87daa61ce8e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:00.367601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 423/860] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:55:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:00.373275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9c98f-c6ec-45ab-b7cd-b87daa61ce8e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:00.432343 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ebeaf9-1a7f-4677-8cf1-cd818052c31e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:00.482676 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01486d1-09c5-45c2-8d5d-587c15520db0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:00.483414 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01486d1-09c5-45c2-8d5d-587c15520db0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:00.483707 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01486d1-09c5-45c2-8d5d-587c15520db0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:00.484066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01486d1-09c5-45c2-8d5d-587c15520db0 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:00.484872 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01486d1-09c5-45c2-8d5d-587c15520db0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:00.485196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01486d1-09c5-45c2-8d5d-587c15520db0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:00.487513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01486d1-09c5-45c2-8d5d-587c15520db0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:00.498172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fde9c98f-c6ec-45ab-b7cd-b87daa61ce8e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:00.507981 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 438/861] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:55:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:00.511463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01486d1-09c5-45c2-8d5d-587c15520db0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:00.538774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f81f5e-5175-4028-a0f3-4adcaf9d9180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:00.539867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f81f5e-5175-4028-a0f3-4adcaf9d9180 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:00.540141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f81f5e-5175-4028-a0f3-4adcaf9d9180 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:00.540425 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22f81f5e-5175-4028-a0f3-4adcaf9d9180 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:00.550931 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f81f5e-5175-4028-a0f3-4adcaf9d9180 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:00.552768 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22f81f5e-5175-4028-a0f3-4adcaf9d9180 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:00.557975 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:00.570072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-22f81f5e-5175-4028-a0f3-4adcaf9d9180 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 15:55:00.570072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:00.570072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:00.570072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:00.570072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:00.570072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:00.570072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:00.570072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:00.570072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:00.570072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:00.570072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:00.570072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:00.570072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:55:00.570072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:55:00.570072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:55:00.570072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:00.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:00.572578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:00.572578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:00.572578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:55:00.572578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:55:00.572578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:55:00.572578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:55:00.572578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:55:00.572578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:55:00.572578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 15:55:00.572578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:00.579559 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 424/862] 10.222.0.85 () {56 vars in 1117 bytes} [Wed May 25 15:55:00 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:00.581368 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f81f5e-5175-4028-a0f3-4adcaf9d9180 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:00.617979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:00.639328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01486d1-09c5-45c2-8d5d-587c15520db0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:00.674536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6a0bd-3228-456a-864b-ae2bb76458b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:00.675611 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6a0bd-3228-456a-864b-ae2bb76458b5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:00.676198 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6a0bd-3228-456a-864b-ae2bb76458b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:00.676697 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb6a0bd-3228-456a-864b-ae2bb76458b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:00.677855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6a0bd-3228-456a-864b-ae2bb76458b5 None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=77fda27641c84fccb68a5cdaf4e8dc4b) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:00.678382 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6a0bd-3228-456a-864b-ae2bb76458b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:00.680124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb6a0bd-3228-456a-864b-ae2bb76458b5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:00.692172 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 439/863] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:55:00 2022] GET /identity/v3/role_assignments?user.id=77fda27641c84fccb68a5cdaf4e8dc4b&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:00.700588 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22f81f5e-5175-4028-a0f3-4adcaf9d9180 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:00.743175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31b248-f29c-4384-ac5b-24298ed2eee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:00.743952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31b248-f29c-4384-ac5b-24298ed2eee1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:00.744246 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31b248-f29c-4384-ac5b-24298ed2eee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:00.744528 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc31b248-f29c-4384-ac5b-24298ed2eee1 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:00.745428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31b248-f29c-4384-ac5b-24298ed2eee1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:00.745761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31b248-f29c-4384-ac5b-24298ed2eee1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:00.747256 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc31b248-f29c-4384-ac5b-24298ed2eee1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:00.769055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 425/864] 10.222.0.85 () {56 vars in 1149 bytes} [Wed May 25 15:55:00 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:00.775004 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6a0bd-3228-456a-864b-ae2bb76458b5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:00.882883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:00.887388 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb6a0bd-3228-456a-864b-ae2bb76458b5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:00.924349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba42eb8b-a0ad-4db9-86fd-3e7d63cc3f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:00.925013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42eb8b-a0ad-4db9-86fd-3e7d63cc3f74 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:00.925309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42eb8b-a0ad-4db9-86fd-3e7d63cc3f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:00.925598 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba42eb8b-a0ad-4db9-86fd-3e7d63cc3f74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:01.002253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42eb8b-a0ad-4db9-86fd-3e7d63cc3f74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:01.003730 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba42eb8b-a0ad-4db9-86fd-3e7d63cc3f74 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:01.124206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:01.127646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 440/865] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:55:00 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:55:01.129230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31b248-f29c-4384-ac5b-24298ed2eee1 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:01.220790 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3d9b2-7f4f-43cd-82f6-9a11f5b89ba1 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:01.221144 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3d9b2-7f4f-43cd-82f6-9a11f5b89ba1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:01.221365 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e3d9b2-7f4f-43cd-82f6-9a11f5b89ba1 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:01.223002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 441/866] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:55:01.227680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:01.227922 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85b347-09e0-4e57-a64f-377b7ff01bfe None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:01.228609 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85b347-09e0-4e57-a64f-377b7ff01bfe None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:01.228900 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c85b347-09e0-4e57-a64f-377b7ff01bfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:01.235525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc31b248-f29c-4384-ac5b-24298ed2eee1 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:01.248670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8c85b347-09e0-4e57-a64f-377b7ff01bfe None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:01.249974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8c85b347-09e0-4e57-a64f-377b7ff01bfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:01.277874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60442061-45d4-4a9a-b306-ff9b3f1bea02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:01.279650 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60442061-45d4-4a9a-b306-ff9b3f1bea02 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:01.280310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60442061-45d4-4a9a-b306-ff9b3f1bea02 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:01.280898 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60442061-45d4-4a9a-b306-ff9b3f1bea02 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:01.304716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60442061-45d4-4a9a-b306-ff9b3f1bea02 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:01.306422 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60442061-45d4-4a9a-b306-ff9b3f1bea02 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:01.326888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-60442061-45d4-4a9a-b306-ff9b3f1bea02 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:01.326888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:01.326888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:01.326888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:01.326888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:01.326888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:01.326888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:01.326888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:01.326888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:01.326888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:01.326888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:01.326888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:01.326888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:55:01.326888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:55:01.326888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:55:01.327980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:55:01.327980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:01.327980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:01.327980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:01.327980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:01.327980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:01.327980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:01.327980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:01.327980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:01.327980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:01.327980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:01.327980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:01.327980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:01.327980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:01.327980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:01.327980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:01.327980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:01.329212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:55:01.329212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:55:01.329212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:01.329212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:01.329469 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 442/867] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:01 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:55:01.335999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 426/868] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:01.337587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85b347-09e0-4e57-a64f-377b7ff01bfe None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:01.350423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60442061-45d4-4a9a-b306-ff9b3f1bea02 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:01.465131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c85b347-09e0-4e57-a64f-377b7ff01bfe None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:01.477450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60442061-45d4-4a9a-b306-ff9b3f1bea02 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:01.494697 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:01.504766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf745ecb-1af9-42c2-b752-1d2393266720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:01.506358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf745ecb-1af9-42c2-b752-1d2393266720 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:01.506843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf745ecb-1af9-42c2-b752-1d2393266720 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:01.507556 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf745ecb-1af9-42c2-b752-1d2393266720 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:01.508912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf745ecb-1af9-42c2-b752-1d2393266720 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:01.509666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf745ecb-1af9-42c2-b752-1d2393266720 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:01.511468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf745ecb-1af9-42c2-b752-1d2393266720 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:01.513436 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efe9f9-41ca-401b-ac7f-ef25d6564e2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:01.514186 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efe9f9-41ca-401b-ac7f-ef25d6564e2f None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:01.514543 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efe9f9-41ca-401b-ac7f-ef25d6564e2f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:01.515138 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27efe9f9-41ca-401b-ac7f-ef25d6564e2f None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:01.515696 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27efe9f9-41ca-401b-ac7f-ef25d6564e2f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:01.517172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27efe9f9-41ca-401b-ac7f-ef25d6564e2f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:01.526494 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 443/869] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:55:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:01.532398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf745ecb-1af9-42c2-b752-1d2393266720 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:01.539260 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 427/870] 10.222.0.85 () {60 vars in 1141 bytes} [Wed May 25 15:55:01 2022] POST /identity/v3/services => generated 251 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:55:01.653759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf745ecb-1af9-42c2-b752-1d2393266720 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:01.698701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710327c-3257-46cc-823e-0b3144c35b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:01.699882 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710327c-3257-46cc-823e-0b3144c35b9b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:01.700383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710327c-3257-46cc-823e-0b3144c35b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:01.700851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e710327c-3257-46cc-823e-0b3144c35b9b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:01.709321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710327c-3257-46cc-823e-0b3144c35b9b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:01.711165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e710327c-3257-46cc-823e-0b3144c35b9b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:01.727262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e710327c-3257-46cc-823e-0b3144c35b9b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:01.727262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:01.727262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:01.727262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:01.727262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:01.727262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:01.727262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:01.727262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:01.727262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:01.727262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:01.727262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:01.727262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:01.727262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:55:01.727262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:55:01.727262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:55:01.728299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:55:01.728299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:01.728299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:01.728299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:01.728299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:01.728299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:01.728299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:01.728299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:01.728299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:01.728299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:01.728299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:01.728299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:01.728299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:01.728299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:01.728299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:01.728299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:01.728299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:01.729231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:55:01.729231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:55:01.729231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:55:01.729231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:55:01.729231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:55:01.729231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:55:01.729231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:01.729231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:01.731592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 444/871] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:55:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:01.737184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efe9f9-41ca-401b-ac7f-ef25d6564e2f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:01.749717 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:01.846891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27efe9f9-41ca-401b-ac7f-ef25d6564e2f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:01.882058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db035d3-83df-4d92-9e25-43a1c055cf23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:01.882736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db035d3-83df-4d92-9e25-43a1c055cf23 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:01.883030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db035d3-83df-4d92-9e25-43a1c055cf23 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:01.883318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db035d3-83df-4d92-9e25-43a1c055cf23 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:01.884148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db035d3-83df-4d92-9e25-43a1c055cf23 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:01.884494 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db035d3-83df-4d92-9e25-43a1c055cf23 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:01.885949 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db035d3-83df-4d92-9e25-43a1c055cf23 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:01.899418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 428/872] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:55:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:01.905733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710327c-3257-46cc-823e-0b3144c35b9b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:01.961558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:02.016560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e710327c-3257-46cc-823e-0b3144c35b9b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:02.051562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d153c8c-e4d9-49e4-9e5e-ae3f07fab719 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:02.052260 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d153c8c-e4d9-49e4-9e5e-ae3f07fab719 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:02.052545 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d153c8c-e4d9-49e4-9e5e-ae3f07fab719 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:02.052826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d153c8c-e4d9-49e4-9e5e-ae3f07fab719 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:02.053668 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d153c8c-e4d9-49e4-9e5e-ae3f07fab719 None admin] RBAC: Adding query filter params (role.id=83bf7d9c29d4464dab8030f6d706cd6d, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=066de1f11f724eee9df0377beb58d92c) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:02.053983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d153c8c-e4d9-49e4-9e5e-ae3f07fab719 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:02.055431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d153c8c-e4d9-49e4-9e5e-ae3f07fab719 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:02.061946 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 445/873] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:55:01 2022] GET /identity/v3/role_assignments?user.id=066de1f11f724eee9df0377beb58d92c&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=83bf7d9c29d4464dab8030f6d706cd6d => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:02.448284 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d50b7f-cfeb-4e22-9721-ad47b866dfa3 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:02.448673 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d50b7f-cfeb-4e22-9721-ad47b866dfa3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:02.448888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5d50b7f-cfeb-4e22-9721-ad47b866dfa3 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:02.450540 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 429/874] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:02.456241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02feff-ba8f-4e4a-bade-4ceee8627df8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:02.456551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02feff-ba8f-4e4a-bade-4ceee8627df8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:02.456793 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f02feff-ba8f-4e4a-bade-4ceee8627df8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:02.473876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1f02feff-ba8f-4e4a-bade-4ceee8627df8 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:02.475069 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1f02feff-ba8f-4e4a-bade-4ceee8627df8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:02.541803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 446/875] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:02 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:02.564650 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d50b7f-cfeb-4e22-9721-ad47b866dfa3 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:02.633273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:02.674299 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5d50b7f-cfeb-4e22-9721-ad47b866dfa3 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:02.708763 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb082876-b941-45a7-a73b-6fdd1623e68c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:02.709781 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb082876-b941-45a7-a73b-6fdd1623e68c None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:02.710299 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb082876-b941-45a7-a73b-6fdd1623e68c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:02.710864 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb082876-b941-45a7-a73b-6fdd1623e68c None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:02.711887 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb082876-b941-45a7-a73b-6fdd1623e68c None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:02.713480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb082876-b941-45a7-a73b-6fdd1623e68c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:02.731617 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 430/876] 10.222.0.85 () {60 vars in 1141 bytes} [Wed May 25 15:55:02 2022] POST /identity/v3/services => generated 269 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:55:03.010550 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005c444-993a-42b7-813a-c012a310b319 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:03.011182 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005c444-993a-42b7-813a-c012a310b319 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:03.011182 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3005c444-993a-42b7-813a-c012a310b319 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:03.012908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 447/877] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:03.018786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cbf95-e6a6-461b-83d7-52a8f2087b36 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:03.019202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cbf95-e6a6-461b-83d7-52a8f2087b36 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:03.019405 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64cbf95-e6a6-461b-83d7-52a8f2087b36 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:03.036474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d64cbf95-e6a6-461b-83d7-52a8f2087b36 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:03.037655 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d64cbf95-e6a6-461b-83d7-52a8f2087b36 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:03.112849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 431/878] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:55:03.134461 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005c444-993a-42b7-813a-c012a310b319 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:03.178883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:03.241110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3005c444-993a-42b7-813a-c012a310b319 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:03.276910 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b21ab0-7571-4200-b757-148860cec676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:03.277650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b21ab0-7571-4200-b757-148860cec676 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:03.277995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b21ab0-7571-4200-b757-148860cec676 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:03.278266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b21ab0-7571-4200-b757-148860cec676 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:03.278973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b21ab0-7571-4200-b757-148860cec676 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:03.280394 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2b21ab0-7571-4200-b757-148860cec676 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:03.294533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a2b21ab0-7571-4200-b757-148860cec676 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 15:55:03.294533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:03.294533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:55:03.294533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:55:03.294533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:55:03.294533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:55:03.294533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:55:03.294533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:03.294533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 15:55:03.294533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:03.294533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:03.294533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:03.294533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:03.294533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:03.294533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:03.294533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:03.294533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:03.295650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:03.295650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:03.295650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:03.295650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:03.295650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:03.295650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:03.295650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:55:03.295650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:55:03.295650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:55:03.295650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:55:03.295650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:03.295650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:03.295650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:03.295650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:03.295650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:03.295650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:03.297005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:03.297005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:03.297005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:03.297005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:03.297005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:03.297005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:03.297005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:03.297005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:03.297005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:03.297005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:03.297005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:55:03.297005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:03.297005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 15:55:03.297005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:03.304087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 448/879] 10.222.0.85 () {56 vars in 1126 bytes} [Wed May 25 15:55:03 2022] GET /identity/v3/services/object-store => generated 93 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:03.307585 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64cbf95-e6a6-461b-83d7-52a8f2087b36 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:03.362027 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:03.412820 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64cbf95-e6a6-461b-83d7-52a8f2087b36 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:03.447731 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ff3c3a-5681-4eaa-a695-796a59d37549 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:03.448546 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff3c3a-5681-4eaa-a695-796a59d37549 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:03.448844 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff3c3a-5681-4eaa-a695-796a59d37549 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:03.449118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ff3c3a-5681-4eaa-a695-796a59d37549 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:03.449882 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff3c3a-5681-4eaa-a695-796a59d37549 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:03.450197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff3c3a-5681-4eaa-a695-796a59d37549 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:03.451577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ff3c3a-5681-4eaa-a695-796a59d37549 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:03.457144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 432/880] 10.222.0.85 () {56 vars in 1122 bytes} [Wed May 25 15:55:03 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:03.462491 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b21ab0-7571-4200-b757-148860cec676 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:03.568531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b21ab0-7571-4200-b757-148860cec676 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:03.570081 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:03.604443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d765ad77-7bf1-45fb-9aa6-9e718e986ab7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:03.605246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765ad77-7bf1-45fb-9aa6-9e718e986ab7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:03.605533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765ad77-7bf1-45fb-9aa6-9e718e986ab7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:03.605814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d765ad77-7bf1-45fb-9aa6-9e718e986ab7 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:03.606525 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765ad77-7bf1-45fb-9aa6-9e718e986ab7 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:03.606838 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765ad77-7bf1-45fb-9aa6-9e718e986ab7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:03.608246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d765ad77-7bf1-45fb-9aa6-9e718e986ab7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:03.613395 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 449/881] 10.222.0.85 () {56 vars in 1122 bytes} [Wed May 25 15:55:03 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:03.915981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da76e9-b278-41fd-ac1f-7e2f920b34ef None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:03.916270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da76e9-b278-41fd-ac1f-7e2f920b34ef None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:03.916807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70da76e9-b278-41fd-ac1f-7e2f920b34ef None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:03.918721 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 433/882] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:03 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:03.924343 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a2d74-4498-46d8-a25f-8ad24eca8fb7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:03.924874 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a2d74-4498-46d8-a25f-8ad24eca8fb7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:03.925405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45a2d74-4498-46d8-a25f-8ad24eca8fb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:03.942750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f45a2d74-4498-46d8-a25f-8ad24eca8fb7 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:03.944317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f45a2d74-4498-46d8-a25f-8ad24eca8fb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:04.014969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 450/883] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:03 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:04.037412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da76e9-b278-41fd-ac1f-7e2f920b34ef None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:04.147514 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70da76e9-b278-41fd-ac1f-7e2f920b34ef None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:04.164596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:04.182799 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab2e52-a52b-4572-8092-cf670bb750cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:04.183545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab2e52-a52b-4572-8092-cf670bb750cf None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:04.183847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab2e52-a52b-4572-8092-cf670bb750cf None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:04.184135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79ab2e52-a52b-4572-8092-cf670bb750cf None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:04.184791 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab2e52-a52b-4572-8092-cf670bb750cf None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:04.186172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79ab2e52-a52b-4572-8092-cf670bb750cf None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:04.198005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-79ab2e52-a52b-4572-8092-cf670bb750cf None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 15:55:04.198005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:04.198005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:55:04.198005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:55:04.198005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:55:04.198005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:55:04.198005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:55:04.198005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:04.198005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 15:55:04.198005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:04.198005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:04.198005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:04.198005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:04.198005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:04.198005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:04.198005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:04.198005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:04.199209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:04.199209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:04.199209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:04.199209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:04.199209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:04.199209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:04.199209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:55:04.199209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:55:04.199209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:55:04.199209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:55:04.199209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:04.199209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:04.199209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:04.199209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:04.199209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:04.199209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:04.200089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:04.200089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:04.200089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:04.200089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:04.200089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:04.200089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:04.200089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:04.200089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:04.200089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:04.200089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:04.200089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:55:04.200089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:04.200089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 15:55:04.200089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:04.200089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 434/884] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:55:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:04.207746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a2d74-4498-46d8-a25f-8ad24eca8fb7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:04.314411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45a2d74-4498-46d8-a25f-8ad24eca8fb7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:04.316126 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:04.350459 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fab595e7-2172-4316-8799-78feaf737d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:04.351139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab595e7-2172-4316-8799-78feaf737d53 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:04.351424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab595e7-2172-4316-8799-78feaf737d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:04.351704 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fab595e7-2172-4316-8799-78feaf737d53 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:04.352460 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab595e7-2172-4316-8799-78feaf737d53 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:04.352771 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab595e7-2172-4316-8799-78feaf737d53 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:04.354155 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fab595e7-2172-4316-8799-78feaf737d53 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:04.359269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 451/885] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:55:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:04.368796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab2e52-a52b-4572-8092-cf670bb750cf None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:04.441529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d2acd-427e-4205-889f-e39d7523f5c6 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:04.441529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d2acd-427e-4205-889f-e39d7523f5c6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:04.441529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30d2acd-427e-4205-889f-e39d7523f5c6 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:04.442611 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 452/886] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:04.447655 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa8cd0-9308-4c8e-bb30-1d417b93a6e0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:04.447987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa8cd0-9308-4c8e-bb30-1d417b93a6e0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:04.448308 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffa8cd0-9308-4c8e-bb30-1d417b93a6e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:04.466351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1ffa8cd0-9308-4c8e-bb30-1d417b93a6e0 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:04.467760 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1ffa8cd0-9308-4c8e-bb30-1d417b93a6e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:04.477376 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79ab2e52-a52b-4572-8092-cf670bb750cf None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:04.519253 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb136640-9085-402e-9865-40d50f8f4270 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:04.519961 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb136640-9085-402e-9865-40d50f8f4270 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:04.520402 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb136640-9085-402e-9865-40d50f8f4270 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:04.520690 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb136640-9085-402e-9865-40d50f8f4270 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:04.521704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb136640-9085-402e-9865-40d50f8f4270 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:04.522022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb136640-9085-402e-9865-40d50f8f4270 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:04.523413 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb136640-9085-402e-9865-40d50f8f4270 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:04.530708 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 435/887] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:55:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:04.536020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb136640-9085-402e-9865-40d50f8f4270 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:04.547940 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 453/888] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:04 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:55:04.565640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa8cd0-9308-4c8e-bb30-1d417b93a6e0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:04.590581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:04.653162 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb136640-9085-402e-9865-40d50f8f4270 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:04.678239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:04.693702 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffa8cd0-9308-4c8e-bb30-1d417b93a6e0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:04.696858 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7823dac-22ac-413f-b56a-e2227b48d73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:04.697765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7823dac-22ac-413f-b56a-e2227b48d73a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:04.698384 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7823dac-22ac-413f-b56a-e2227b48d73a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:04.698858 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7823dac-22ac-413f-b56a-e2227b48d73a None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:04.699863 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7823dac-22ac-413f-b56a-e2227b48d73a None admin] RBAC: Adding query filter params (interface=public, service_id=24dd1a6b819046bf971c93ca276b1028, region_id=RegionOne) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:04.700397 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7823dac-22ac-413f-b56a-e2227b48d73a None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:04.702572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7823dac-22ac-413f-b56a-e2227b48d73a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:04.715298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 436/889] 10.222.0.85 () {56 vars in 1251 bytes} [Wed May 25 15:55:04 2022] GET /identity/v3/endpoints?service_id=24dd1a6b819046bf971c93ca276b1028&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:04.716227 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7823dac-22ac-413f-b56a-e2227b48d73a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:04.737571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c4aae-7208-471e-88c3-f1fc04e6a186 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:04.739166 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c4aae-7208-471e-88c3-f1fc04e6a186 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:04.739652 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c4aae-7208-471e-88c3-f1fc04e6a186 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:04.740414 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013c4aae-7208-471e-88c3-f1fc04e6a186 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:04.753983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013c4aae-7208-471e-88c3-f1fc04e6a186 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:55:04.762895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-013c4aae-7208-471e-88c3-f1fc04e6a186 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 15:55:04.762895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:04.762895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:04.762895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:04.762895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:04.762895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:04.762895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:04.762895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:04.762895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:04.762895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:04.762895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:04.762895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:04.762895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:55:04.762895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:55:04.762895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:55:04.762895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:55:04.763924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:55:04.763924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:55:04.763924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:55:04.763924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:55:04.763924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:04.763924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:04.763924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:04.763924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:04.763924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:04.763924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:04.763924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:04.763924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:04.763924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:04.763924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:04.763924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:04.763924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:04.764786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:04.764786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:04.764786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:04.764786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:04.764786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:55:04.764786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:55:04.764786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:55:04.764786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:55:04.764786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:55:04.764786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:55:04.764786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 15:55:04.764786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:04.769100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 454/890] 10.222.0.85 () {56 vars in 1120 bytes} [Wed May 25 15:55:04 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:55:04.770877 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c4aae-7208-471e-88c3-f1fc04e6a186 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:04.846585 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7823dac-22ac-413f-b56a-e2227b48d73a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:04.858452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:04.888830 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013c4aae-7208-471e-88c3-f1fc04e6a186 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:04.891219 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064ea83-7d4c-4dad-a572-bf1cfe05e20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:04.892618 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064ea83-7d4c-4dad-a572-bf1cfe05e20e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:04.892914 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064ea83-7d4c-4dad-a572-bf1cfe05e20e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:04.893201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2064ea83-7d4c-4dad-a572-bf1cfe05e20e None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:04.893900 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064ea83-7d4c-4dad-a572-bf1cfe05e20e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:04.895307 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2064ea83-7d4c-4dad-a572-bf1cfe05e20e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:04.908014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 437/891] 10.222.0.85 () {56 vars in 1087 bytes} [Wed May 25 15:55:04 2022] GET /identity/v3/services => generated 1268 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:55:04.934751 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba9cac0-ea36-4dc4-8011-167d7e4e5df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:04.935899 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9cac0-ea36-4dc4-8011-167d7e4e5df6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:04.936431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9cac0-ea36-4dc4-8011-167d7e4e5df6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:04.936916 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba9cac0-ea36-4dc4-8011-167d7e4e5df6 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:04.937993 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9cac0-ea36-4dc4-8011-167d7e4e5df6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:04.938519 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9cac0-ea36-4dc4-8011-167d7e4e5df6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:04.940262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba9cac0-ea36-4dc4-8011-167d7e4e5df6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:04.950999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 455/892] 10.222.0.85 () {56 vars in 1115 bytes} [Wed May 25 15:55:04 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:04.956979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064ea83-7d4c-4dad-a572-bf1cfe05e20e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:05.011227 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd487726-7121-4a38-97c4-e5a908d74c7d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:05.011670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd487726-7121-4a38-97c4-e5a908d74c7d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:05.012070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd487726-7121-4a38-97c4-e5a908d74c7d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:05.013804 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 456/893] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:55:05.019351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fe9a2-6a05-4fd0-a4bc-ccbb1e9dc095 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:05.020029 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fe9a2-6a05-4fd0-a4bc-ccbb1e9dc095 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:05.020029 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46fe9a2-6a05-4fd0-a4bc-ccbb1e9dc095 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:05.037837 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f46fe9a2-6a05-4fd0-a4bc-ccbb1e9dc095 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:05.039186 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f46fe9a2-6a05-4fd0-a4bc-ccbb1e9dc095 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:05.048013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:05.079207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2064ea83-7d4c-4dad-a572-bf1cfe05e20e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:05.119982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76033bdf-53d1-40cd-b4e2-609ecaa30e5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:05.121046 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76033bdf-53d1-40cd-b4e2-609ecaa30e5b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:05.121452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76033bdf-53d1-40cd-b4e2-609ecaa30e5b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:05.121796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76033bdf-53d1-40cd-b4e2-609ecaa30e5b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:05.135017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76033bdf-53d1-40cd-b4e2-609ecaa30e5b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:05.144590 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76033bdf-53d1-40cd-b4e2-609ecaa30e5b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:05.144766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 457/894] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:05 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:55:05.157665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-76033bdf-53d1-40cd-b4e2-609ecaa30e5b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:05.157665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:05.157665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:05.157665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:05.157665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:05.157665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:05.157665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:05.157665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:05.157665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:05.157665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:05.157665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:05.157665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:05.157665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:55:05.157665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:55:05.157665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:55:05.158679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:55:05.158679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:05.158679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:05.158679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:05.158679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:05.158679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:05.158679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:05.158679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:05.158679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:05.158679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:05.158679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:05.158679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:05.158679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:05.158679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:05.158679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:05.158679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:05.158679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:05.159629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:55:05.159629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:55:05.159629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:05.159629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:05.160769 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 438/895] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:05.164575 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fe9a2-6a05-4fd0-a4bc-ccbb1e9dc095 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:05.169771 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76033bdf-53d1-40cd-b4e2-609ecaa30e5b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:05.203790 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:05.281259 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46fe9a2-6a05-4fd0-a4bc-ccbb1e9dc095 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:05.311202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76033bdf-53d1-40cd-b4e2-609ecaa30e5b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:05.335312 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d31cfa-d8aa-4def-8f49-1e5118f3d463 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:05.337201 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d31cfa-d8aa-4def-8f49-1e5118f3d463 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:05.337675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d31cfa-d8aa-4def-8f49-1e5118f3d463 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:05.338383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d31cfa-d8aa-4def-8f49-1e5118f3d463 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:05.339690 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d31cfa-d8aa-4def-8f49-1e5118f3d463 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:05.341780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d31cfa-d8aa-4def-8f49-1e5118f3d463 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:05.347521 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4ce0a-b72a-4cc7-9807-522faf8816af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:05.348370 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4ce0a-b72a-4cc7-9807-522faf8816af None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:05.348724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4ce0a-b72a-4cc7-9807-522faf8816af None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:05.349081 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4af4ce0a-b72a-4cc7-9807-522faf8816af None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:05.349876 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4ce0a-b72a-4cc7-9807-522faf8816af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:05.350255 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4ce0a-b72a-4cc7-9807-522faf8816af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:05.351728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4af4ce0a-b72a-4cc7-9807-522faf8816af None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:05.358629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-00d31cfa-d8aa-4def-8f49-1e5118f3d463 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 15:55:05.358629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:05.358629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:55:05.358629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:55:05.358629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:55:05.358629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:55:05.358629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:55:05.358629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:05.358629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 15:55:05.358629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:05.358629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:05.358629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:05.358629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:05.358629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:05.358629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:05.358629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:05.359627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:05.359627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:05.359627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:05.359627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:05.359627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:05.359627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:05.359627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:05.359627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:55:05.359627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:55:05.359627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:55:05.359627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:55:05.359627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:05.359627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:05.359627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:05.359627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:05.359627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:05.360534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:05.360534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:05.360534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:05.360534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:05.360534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:05.360534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:05.360534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:05.360534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:05.360534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:05.360534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:05.360534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:05.360534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:55:05.360534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:05.360534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 15:55:05.360534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:05.362552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 458/896] 10.222.0.85 () {56 vars in 1132 bytes} [Wed May 25 15:55:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:05.368795 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d31cfa-d8aa-4def-8f49-1e5118f3d463 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:05.374199 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 439/897] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:55:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:05.376700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4ce0a-b72a-4cc7-9807-522faf8816af None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:05.382568 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:05.462781 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:05.479387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4af4ce0a-b72a-4cc7-9807-522faf8816af None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:05.488297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d31cfa-d8aa-4def-8f49-1e5118f3d463 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:05.513827 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa9b74-cb97-4975-a159-216b2d79575e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:05.514703 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa9b74-cb97-4975-a159-216b2d79575e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:05.514994 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa9b74-cb97-4975-a159-216b2d79575e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:05.515271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4fa9b74-cb97-4975-a159-216b2d79575e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:05.523083 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa9b74-cb97-4975-a159-216b2d79575e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:05.524639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4fa9b74-cb97-4975-a159-216b2d79575e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:05.528189 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1aae2-c69b-4135-a93a-c77857c130f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:05.529124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1aae2-c69b-4135-a93a-c77857c130f5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:05.529615 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1aae2-c69b-4135-a93a-c77857c130f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:05.530113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b1aae2-c69b-4135-a93a-c77857c130f5 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:05.531037 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1aae2-c69b-4135-a93a-c77857c130f5 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:05.531552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1aae2-c69b-4135-a93a-c77857c130f5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:05.533164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b1aae2-c69b-4135-a93a-c77857c130f5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:05.538837 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 459/898] 10.222.0.85 () {56 vars in 1126 bytes} [Wed May 25 15:55:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:05.542292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-d4fa9b74-cb97-4975-a159-216b2d79575e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 15:55:05.542292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:05.542292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:05.542292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:05.542292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:05.542292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:05.542292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:05.542292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:05.542292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:05.542292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:05.542292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:05.542292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:05.542292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:55:05.542292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:55:05.542292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:55:05.542292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:05.543396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:05.544395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:05.544395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:05.544395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:55:05.544395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:55:05.544395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:55:05.544395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:55:05.544395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:55:05.544395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:55:05.544395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 15:55:05.544395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:05.546901 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 440/899] 10.222.0.85 () {56 vars in 1117 bytes} [Wed May 25 15:55:05 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:05.548198 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1aae2-c69b-4135-a93a-c77857c130f5 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:05.549909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa9b74-cb97-4975-a159-216b2d79575e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:05.624515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:05.652091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4fa9b74-cb97-4975-a159-216b2d79575e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:05.659605 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b1aae2-c69b-4135-a93a-c77857c130f5 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:05.688011 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec795c30-e904-423c-8f60-34943d6b3fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:05.688880 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec795c30-e904-423c-8f60-34943d6b3fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:05.689110 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec795c30-e904-423c-8f60-34943d6b3fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:05.689390 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec795c30-e904-423c-8f60-34943d6b3fe2 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:05.690115 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec795c30-e904-423c-8f60-34943d6b3fe2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:05.690409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec795c30-e904-423c-8f60-34943d6b3fe2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:05.691844 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec795c30-e904-423c-8f60-34943d6b3fe2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:05.702920 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 441/900] 10.222.0.85 () {56 vars in 1126 bytes} [Wed May 25 15:55:05 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:05.703158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2216ae-0f3e-455c-8aaa-a65c65801e15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:05.703158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2216ae-0f3e-455c-8aaa-a65c65801e15 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:05.703158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2216ae-0f3e-455c-8aaa-a65c65801e15 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:05.703158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be2216ae-0f3e-455c-8aaa-a65c65801e15 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:05.703454 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2216ae-0f3e-455c-8aaa-a65c65801e15 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:05.703454 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2216ae-0f3e-455c-8aaa-a65c65801e15 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:05.704470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be2216ae-0f3e-455c-8aaa-a65c65801e15 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:05.705589 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec795c30-e904-423c-8f60-34943d6b3fe2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:05.721269 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:05.735916 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 460/901] 10.222.0.85 () {56 vars in 1149 bytes} [Wed May 25 15:55:05 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:05.738560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2216ae-0f3e-455c-8aaa-a65c65801e15 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:05.836493 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec795c30-e904-423c-8f60-34943d6b3fe2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:05.855261 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be2216ae-0f3e-455c-8aaa-a65c65801e15 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:05.872069 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebf92c-3a84-4e9e-a28d-016f6e239a70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:05.872838 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebf92c-3a84-4e9e-a28d-016f6e239a70 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:05.873129 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebf92c-3a84-4e9e-a28d-016f6e239a70 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:05.873408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bebf92c-3a84-4e9e-a28d-016f6e239a70 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:05.874195 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bebf92c-3a84-4e9e-a28d-016f6e239a70 None admin] RBAC: Adding query filter params (interface=public, service_id=8b43b746c78746829c46a9f451bc4eee, region_id=RegionOne) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:05.874506 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bebf92c-3a84-4e9e-a28d-016f6e239a70 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:05.876003 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bebf92c-3a84-4e9e-a28d-016f6e239a70 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:05.885853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 442/902] 10.222.0.85 () {56 vars in 1251 bytes} [Wed May 25 15:55:05 2022] GET /identity/v3/endpoints?service_id=8b43b746c78746829c46a9f451bc4eee&interface=public®ion_id=RegionOne => generated 194 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:05.888217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebf92c-3a84-4e9e-a28d-016f6e239a70 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:05.892488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:05.901811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf00f3-826a-44b8-a578-930050209a06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:05.902600 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf00f3-826a-44b8-a578-930050209a06 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:05.903012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf00f3-826a-44b8-a578-930050209a06 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:05.903396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcbf00f3-826a-44b8-a578-930050209a06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:05.959668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:05.972840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf00f3-826a-44b8-a578-930050209a06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:05.974236 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcbf00f3-826a-44b8-a578-930050209a06 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:05.992754 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bebf92c-3a84-4e9e-a28d-016f6e239a70 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:06.027869 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450ddca1-a608-4903-9ce5-068438b1dcc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:06.028569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ddca1-a608-4903-9ce5-068438b1dcc7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:06.028990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ddca1-a608-4903-9ce5-068438b1dcc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:06.029353 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450ddca1-a608-4903-9ce5-068438b1dcc7 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:06.030055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ddca1-a608-4903-9ce5-068438b1dcc7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:06.031526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450ddca1-a608-4903-9ce5-068438b1dcc7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:06.038165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 443/903] 10.222.0.85 () {56 vars in 1087 bytes} [Wed May 25 15:55:05 2022] GET /identity/v3/services => generated 1268 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:55:06.039254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1fb21-1c23-442c-a4a6-d1698f10ce4e None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:06.039542 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1fb21-1c23-442c-a4a6-d1698f10ce4e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:06.039814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc1fb21-1c23-442c-a4a6-d1698f10ce4e None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:06.041188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 444/904] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:06.046737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c753ad-7bd0-4743-aebb-4950b2db8eb5 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:06.047023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c753ad-7bd0-4743-aebb-4950b2db8eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:06.047304 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c753ad-7bd0-4743-aebb-4950b2db8eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:06.066375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-24c753ad-7bd0-4743-aebb-4950b2db8eb5 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:06.067675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-24c753ad-7bd0-4743-aebb-4950b2db8eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:06.107903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 461/905] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:55:05 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 365 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:55:06.109203 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf00f3-826a-44b8-a578-930050209a06 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:06.141180 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:06.172765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 445/906] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:06 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:55:06.198638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c753ad-7bd0-4743-aebb-4950b2db8eb5 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:06.234964 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcbf00f3-826a-44b8-a578-930050209a06 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:06.271879 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:06.272427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c749a3-0435-4b7e-972d-8c1d4c5314c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:06.273312 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c749a3-0435-4b7e-972d-8c1d4c5314c3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:06.273537 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c749a3-0435-4b7e-972d-8c1d4c5314c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:06.273811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c749a3-0435-4b7e-972d-8c1d4c5314c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:06.288394 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c749a3-0435-4b7e-972d-8c1d4c5314c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:06.289873 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c749a3-0435-4b7e-972d-8c1d4c5314c3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:06.309431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a1c749a3-0435-4b7e-972d-8c1d4c5314c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:06.309431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:06.309431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:06.309431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:06.309431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:06.309431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:06.309431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:06.309431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:06.309431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:06.309431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:06.309431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:06.309431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:06.309431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:55:06.309431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:55:06.309431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:55:06.310351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:55:06.310351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:06.310351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:06.310351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:06.310351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:06.310351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:06.310351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:06.310351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:06.310351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:06.310351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:06.310351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:06.310351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:06.310351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:06.310351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:06.310351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:06.310351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:06.310351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:06.311242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:55:06.311242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:55:06.311242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:06.311242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:06.311242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 462/907] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:06.319858 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c753ad-7bd0-4743-aebb-4950b2db8eb5 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:06.320037 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c749a3-0435-4b7e-972d-8c1d4c5314c3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:06.350634 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af52d8-58c2-4557-9c07-3bb2165f93c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:06.351442 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af52d8-58c2-4557-9c07-3bb2165f93c0 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:06.351790 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af52d8-58c2-4557-9c07-3bb2165f93c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:06.352197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0af52d8-58c2-4557-9c07-3bb2165f93c0 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:06.353000 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af52d8-58c2-4557-9c07-3bb2165f93c0 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:06.354411 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0af52d8-58c2-4557-9c07-3bb2165f93c0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:06.371988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 446/908] 10.222.0.85 () {60 vars in 1141 bytes} [Wed May 25 15:55:06 2022] POST /identity/v3/services => generated 241 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:55:06.415223 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:06.440203 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c749a3-0435-4b7e-972d-8c1d4c5314c3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:06.490903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58753e75-f60f-4a78-9f64-10ab4a5e011f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:06.491509 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58753e75-f60f-4a78-9f64-10ab4a5e011f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:06.491799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58753e75-f60f-4a78-9f64-10ab4a5e011f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:06.492142 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58753e75-f60f-4a78-9f64-10ab4a5e011f None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:06.492891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58753e75-f60f-4a78-9f64-10ab4a5e011f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:06.493212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58753e75-f60f-4a78-9f64-10ab4a5e011f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:06.494659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58753e75-f60f-4a78-9f64-10ab4a5e011f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:06.508644 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 463/909] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:55:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:06.517027 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af52d8-58c2-4557-9c07-3bb2165f93c0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:06.628432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0af52d8-58c2-4557-9c07-3bb2165f93c0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:06.655184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:06.664340 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9bcf4-4210-4261-bce0-87342e415201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:06.665413 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9bcf4-4210-4261-bce0-87342e415201 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:06.666076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9bcf4-4210-4261-bce0-87342e415201 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:06.666368 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a9bcf4-4210-4261-bce0-87342e415201 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:06.674024 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9bcf4-4210-4261-bce0-87342e415201 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:06.675550 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a9bcf4-4210-4261-bce0-87342e415201 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:06.690241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-76a9bcf4-4210-4261-bce0-87342e415201 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:06.690241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:06.690241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:06.690241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:06.690241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:06.690241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:06.690241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:06.690241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:06.690241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:06.690241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:06.690241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:06.690241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:06.690241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:55:06.690241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:55:06.690241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:55:06.691132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:55:06.691132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:06.691132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:06.691132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:06.691132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:06.691132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:06.691132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:06.691132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:06.691132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:06.691132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:06.691132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:06.691132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:06.691132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:06.691132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:06.691132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:06.691132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:06.691132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:06.692066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:55:06.692066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:55:06.692066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:55:06.692066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:55:06.692066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:55:06.692066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:55:06.692066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:06.692066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:06.692066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 447/910] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:55:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:06.697600 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58753e75-f60f-4a78-9f64-10ab4a5e011f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:06.802417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58753e75-f60f-4a78-9f64-10ab4a5e011f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:06.837826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9634895d-c001-49b3-ae02-bbe5bce32fa8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:06.838504 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634895d-c001-49b3-ae02-bbe5bce32fa8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:06.838845 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634895d-c001-49b3-ae02-bbe5bce32fa8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:06.839134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9634895d-c001-49b3-ae02-bbe5bce32fa8 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:06.840146 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634895d-c001-49b3-ae02-bbe5bce32fa8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:06.840468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634895d-c001-49b3-ae02-bbe5bce32fa8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:06.841934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9634895d-c001-49b3-ae02-bbe5bce32fa8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:06.844786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:06.855454 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 464/911] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:55:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:06.861452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9bcf4-4210-4261-bce0-87342e415201 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:06.961988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a9bcf4-4210-4261-bce0-87342e415201 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:06.997081 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7bf96e5-b778-4d9a-9df3-ad2f78b0ac64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:06.997754 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf96e5-b778-4d9a-9df3-ad2f78b0ac64 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:06.998045 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf96e5-b778-4d9a-9df3-ad2f78b0ac64 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:06.998327 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7bf96e5-b778-4d9a-9df3-ad2f78b0ac64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:06.999174 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf96e5-b778-4d9a-9df3-ad2f78b0ac64 None admin] RBAC: Adding query filter params (role.id=f1cd5d80f39b48bdb388dd5c539d9a0b, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=066de1f11f724eee9df0377beb58d92c) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:06.999499 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf96e5-b778-4d9a-9df3-ad2f78b0ac64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:07.000991 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7bf96e5-b778-4d9a-9df3-ad2f78b0ac64 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:07.007211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 448/912] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:55:06 2022] GET /identity/v3/role_assignments?user.id=066de1f11f724eee9df0377beb58d92c&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=f1cd5d80f39b48bdb388dd5c539d9a0b => generated 259 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:07.241348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d445ac4-bd2f-406a-9938-06a4b19aa912 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:07.242059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d445ac4-bd2f-406a-9938-06a4b19aa912 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:07.242059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d445ac4-bd2f-406a-9938-06a4b19aa912 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:07.243512 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 465/913] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:07.249025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfbee8-538b-4804-8cce-1cef56548cda None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:07.249457 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfbee8-538b-4804-8cce-1cef56548cda None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:07.249634 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0bfbee8-538b-4804-8cce-1cef56548cda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:07.269305 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e0bfbee8-538b-4804-8cce-1cef56548cda None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:07.270829 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e0bfbee8-538b-4804-8cce-1cef56548cda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:07.335026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:07.358812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 449/914] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:07 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:07.381197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d445ac4-bd2f-406a-9938-06a4b19aa912 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:07.488651 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d445ac4-bd2f-406a-9938-06a4b19aa912 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:07.523009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c603360-05a4-4202-80d9-4d495f083c73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:07.523750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c603360-05a4-4202-80d9-4d495f083c73 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:07.524098 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c603360-05a4-4202-80d9-4d495f083c73 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:07.524375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c603360-05a4-4202-80d9-4d495f083c73 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:07.525033 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c603360-05a4-4202-80d9-4d495f083c73 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:07.526464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c603360-05a4-4202-80d9-4d495f083c73 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:07.538296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9c603360-05a4-4202-80d9-4d495f083c73 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 15:55:07.538296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:07.538296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:55:07.538296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:55:07.538296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:55:07.538296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:55:07.538296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:55:07.538296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:07.538296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 15:55:07.538296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:07.538296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:07.538296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:07.538296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:07.538296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:07.538296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:07.538296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:07.538296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:07.539306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:07.539306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:07.539306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:07.539306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:07.539306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:07.539306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:07.539306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:55:07.539306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:55:07.539306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:55:07.539306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:55:07.539306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:07.539306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:07.539306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:07.539306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:07.539306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:07.539306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:07.540216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:07.540216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:07.540216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:07.540216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:07.540216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:07.540216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:07.540216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:07.540216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:07.540216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:07.540216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:07.540216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:55:07.540216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:07.540216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. May 25 15:55:07.540216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:07.540216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 466/915] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:55:07 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:07.547707 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfbee8-538b-4804-8cce-1cef56548cda None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:07.654854 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0bfbee8-538b-4804-8cce-1cef56548cda None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:07.689831 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbad39-868a-4b32-8240-508a15f2166c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:07.690521 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbad39-868a-4b32-8240-508a15f2166c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:07.690814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbad39-868a-4b32-8240-508a15f2166c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:07.691098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18dbad39-868a-4b32-8240-508a15f2166c None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:07.691840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dbad39-868a-4b32-8240-508a15f2166c None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:07.692172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dbad39-868a-4b32-8240-508a15f2166c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:07.693578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18dbad39-868a-4b32-8240-508a15f2166c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:07.703379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 450/916] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:55:07 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:07.704824 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c603360-05a4-4202-80d9-4d495f083c73 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:07.766908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:07.815174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c603360-05a4-4202-80d9-4d495f083c73 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:07.850268 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df39dfee-7e0b-46ab-bba6-17578c0a8ba5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:07.850946 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39dfee-7e0b-46ab-bba6-17578c0a8ba5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:07.851244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39dfee-7e0b-46ab-bba6-17578c0a8ba5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:07.851527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df39dfee-7e0b-46ab-bba6-17578c0a8ba5 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:07.852276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39dfee-7e0b-46ab-bba6-17578c0a8ba5 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:07.852591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39dfee-7e0b-46ab-bba6-17578c0a8ba5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:07.853983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df39dfee-7e0b-46ab-bba6-17578c0a8ba5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:07.859409 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 467/917] 10.222.0.85 () {56 vars in 1114 bytes} [Wed May 25 15:55:07 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:07.864993 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbad39-868a-4b32-8240-508a15f2166c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:07.912202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:07.972278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18dbad39-868a-4b32-8240-508a15f2166c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:08.007110 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20641da9-b5dc-4e7a-a32b-491df8ed988b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:08.008211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20641da9-b5dc-4e7a-a32b-491df8ed988b None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:08.008702 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20641da9-b5dc-4e7a-a32b-491df8ed988b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:08.009176 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20641da9-b5dc-4e7a-a32b-491df8ed988b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:08.010123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20641da9-b5dc-4e7a-a32b-491df8ed988b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:08.011769 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20641da9-b5dc-4e7a-a32b-491df8ed988b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:08.032765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 451/918] 10.222.0.85 () {60 vars in 1144 bytes} [Wed May 25 15:55:07 2022] POST /identity/v3/endpoints => generated 346 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:55:08.392513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a5b74-3d27-40a7-a382-fabc965ed0f6 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:08.393508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a5b74-3d27-40a7-a382-fabc965ed0f6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:08.394069 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a5b74-3d27-40a7-a382-fabc965ed0f6 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:08.395948 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 468/919] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:08.401787 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97feb44-a017-4495-a276-39ac66b9165d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:08.402627 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97feb44-a017-4495-a276-39ac66b9165d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:08.403118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97feb44-a017-4495-a276-39ac66b9165d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:08.421565 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e97feb44-a017-4495-a276-39ac66b9165d None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:08.423191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e97feb44-a017-4495-a276-39ac66b9165d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:08.499261 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 452/920] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:55:08.522476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a5b74-3d27-40a7-a382-fabc965ed0f6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:08.634237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a5b74-3d27-40a7-a382-fabc965ed0f6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:08.658935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:08.671781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5315ed87-43c0-42f3-85fc-de42803e901e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:08.672990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315ed87-43c0-42f3-85fc-de42803e901e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:08.673485 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315ed87-43c0-42f3-85fc-de42803e901e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:08.673962 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5315ed87-43c0-42f3-85fc-de42803e901e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:08.674820 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5315ed87-43c0-42f3-85fc-de42803e901e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:08.676433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5315ed87-43c0-42f3-85fc-de42803e901e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:08.689361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-5315ed87-43c0-42f3-85fc-de42803e901e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 15:55:08.689361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:08.689361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:55:08.689361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:55:08.689361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:55:08.689361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:55:08.689361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:55:08.689361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:08.689361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 15:55:08.689361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:08.689361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:08.689361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:08.689361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:08.689361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:08.689361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:08.689361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:08.690371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:08.690371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:08.690371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:08.690371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:08.690371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:08.690371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:08.690371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:08.690371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:55:08.690371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:55:08.690371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:55:08.690371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:55:08.690371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:08.690371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:08.690371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:08.690371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:08.690371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:08.691245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:08.691245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:08.691245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:08.691245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:08.691245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:08.691245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:08.691245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:08.691245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:08.691245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:08.691245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:08.691245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:08.691245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:55:08.691245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:08.691245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. May 25 15:55:08.691245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:08.694165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 469/921] 10.222.0.85 () {56 vars in 1132 bytes} [Wed May 25 15:55:08 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:08.700561 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97feb44-a017-4495-a276-39ac66b9165d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:08.729821 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:08.775269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b72a-584d-4ca7-91ea-d9339b32ae36 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:08.775925 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b72a-584d-4ca7-91ea-d9339b32ae36 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:08.776456 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f7b72a-584d-4ca7-91ea-d9339b32ae36 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:08.779250 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 470/922] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:08 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:08.785324 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44485cb9-a6d4-4791-b7f3-b884a6b3ac06 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:08.785955 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44485cb9-a6d4-4791-b7f3-b884a6b3ac06 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:08.787463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44485cb9-a6d4-4791-b7f3-b884a6b3ac06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:08.809468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-44485cb9-a6d4-4791-b7f3-b884a6b3ac06 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:08.811681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-44485cb9-a6d4-4791-b7f3-b884a6b3ac06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:08.819768 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97feb44-a017-4495-a276-39ac66b9165d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:08.865316 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab1f73-eaa0-4002-9f24-6d46ce36a60d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:08.866000 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1f73-eaa0-4002-9f24-6d46ce36a60d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:08.866288 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1f73-eaa0-4002-9f24-6d46ce36a60d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:08.866567 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ab1f73-eaa0-4002-9f24-6d46ce36a60d None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:08.867296 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab1f73-eaa0-4002-9f24-6d46ce36a60d None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:08.867611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab1f73-eaa0-4002-9f24-6d46ce36a60d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:08.869049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3ab1f73-eaa0-4002-9f24-6d46ce36a60d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:08.879859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 453/923] 10.222.0.85 () {56 vars in 1126 bytes} [Wed May 25 15:55:08 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:08.882156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab1f73-eaa0-4002-9f24-6d46ce36a60d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:08.912002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 471/924] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:08.926423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44485cb9-a6d4-4791-b7f3-b884a6b3ac06 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:08.986435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:08.998387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3ab1f73-eaa0-4002-9f24-6d46ce36a60d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:09.032673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44485cb9-a6d4-4791-b7f3-b884a6b3ac06 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:09.038981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64019f35-e2a9-4cac-b682-cc205c905e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:09.040459 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64019f35-e2a9-4cac-b682-cc205c905e47 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:09.040943 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64019f35-e2a9-4cac-b682-cc205c905e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:09.041382 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64019f35-e2a9-4cac-b682-cc205c905e47 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:09.042614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64019f35-e2a9-4cac-b682-cc205c905e47 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:09.043260 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64019f35-e2a9-4cac-b682-cc205c905e47 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:09.045470 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64019f35-e2a9-4cac-b682-cc205c905e47 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:09.056037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 454/925] 10.222.0.85 () {56 vars in 1126 bytes} [Wed May 25 15:55:08 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:09.056273 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:09.058467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64019f35-e2a9-4cac-b682-cc205c905e47 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:09.071992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14f312-4440-4faa-8f96-7075277d675b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:09.072817 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14f312-4440-4faa-8f96-7075277d675b None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:09.073140 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14f312-4440-4faa-8f96-7075277d675b None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:09.073500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e14f312-4440-4faa-8f96-7075277d675b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:09.074193 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14f312-4440-4faa-8f96-7075277d675b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:09.075641 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e14f312-4440-4faa-8f96-7075277d675b None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:09.088843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8e14f312-4440-4faa-8f96-7075277d675b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 15:55:09.088843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:09.088843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:55:09.088843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:55:09.088843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:55:09.088843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:55:09.088843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:55:09.088843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:09.088843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 15:55:09.088843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:09.088843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:09.088843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:09.088843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:09.088843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:09.088843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:09.088843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:09.088843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:09.089879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:09.089879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:09.089879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:09.089879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:09.089879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:09.089879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:09.089879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:55:09.089879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:55:09.089879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:55:09.089879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:55:09.089879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:09.089879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:09.089879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:09.089879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:09.089879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:09.089879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:09.090741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:09.090741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:09.090741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:09.090741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:09.090741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:09.090741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:09.090741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:09.090741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:09.090741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:09.090741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:09.090741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:55:09.090741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:09.090741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 15:55:09.090741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:09.095341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 472/926] 10.222.0.85 () {56 vars in 1126 bytes} [Wed May 25 15:55:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:09.096205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14f312-4440-4faa-8f96-7075277d675b None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:09.175336 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64019f35-e2a9-4cac-b682-cc205c905e47 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:09.213307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e14f312-4440-4faa-8f96-7075277d675b None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:09.215372 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93ef2bde-7728-4a41-9376-6c60869bc33b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:09.216089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef2bde-7728-4a41-9376-6c60869bc33b None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:09.216379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef2bde-7728-4a41-9376-6c60869bc33b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:09.216659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93ef2bde-7728-4a41-9376-6c60869bc33b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:09.217412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef2bde-7728-4a41-9376-6c60869bc33b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:09.218795 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93ef2bde-7728-4a41-9376-6c60869bc33b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:09.248114 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 455/927] 10.222.0.85 () {60 vars in 1144 bytes} [Wed May 25 15:55:09 2022] POST /identity/v3/endpoints => generated 347 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:55:09.250168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f93f4-d419-44c5-ad9c-cd0b39d1cd91 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:09.250518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f93f4-d419-44c5-ad9c-cd0b39d1cd91 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:09.250853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3f93f4-d419-44c5-ad9c-cd0b39d1cd91 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:09.256883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 456/928] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:09.258795 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f8e0c-c937-4392-b0d0-a5e895b17cf2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:09.259091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f8e0c-c937-4392-b0d0-a5e895b17cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:09.259372 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3f8e0c-c937-4392-b0d0-a5e895b17cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:09.267557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e453194-ff7e-4a98-830f-79da64108445 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:09.268298 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453194-ff7e-4a98-830f-79da64108445 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:09.268588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453194-ff7e-4a98-830f-79da64108445 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:09.268872 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e453194-ff7e-4a98-830f-79da64108445 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:09.271377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e453194-ff7e-4a98-830f-79da64108445 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:09.271729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e453194-ff7e-4a98-830f-79da64108445 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:09.273225 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e453194-ff7e-4a98-830f-79da64108445 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:09.285277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 473/929] 10.222.0.85 () {56 vars in 1122 bytes} [Wed May 25 15:55:09 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:09.290795 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e453194-ff7e-4a98-830f-79da64108445 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:09.293322 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8f3f8e0c-c937-4392-b0d0-a5e895b17cf2 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:09.294627 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8f3f8e0c-c937-4392-b0d0-a5e895b17cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:09.298635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:09.367225 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:09.387845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 457/930] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:09.409657 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f8e0c-c937-4392-b0d0-a5e895b17cf2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:09.453422 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e453194-ff7e-4a98-830f-79da64108445 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:09.498104 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bcab2c-6571-4f97-85dd-e908e05d8b1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:09.499160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bcab2c-6571-4f97-85dd-e908e05d8b1d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:09.499647 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bcab2c-6571-4f97-85dd-e908e05d8b1d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:09.500533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9bcab2c-6571-4f97-85dd-e908e05d8b1d None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:09.501433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bcab2c-6571-4f97-85dd-e908e05d8b1d None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:09.502293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bcab2c-6571-4f97-85dd-e908e05d8b1d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:09.504471 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9bcab2c-6571-4f97-85dd-e908e05d8b1d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:09.515168 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 474/931] 10.222.0.85 () {56 vars in 1122 bytes} [Wed May 25 15:55:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:09.520729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bcab2c-6571-4f97-85dd-e908e05d8b1d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:09.527190 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3f8e0c-c937-4392-b0d0-a5e895b17cf2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:09.563878 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94faa8cb-b730-4ffe-85cd-bcc44440c7cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:09.564764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94faa8cb-b730-4ffe-85cd-bcc44440c7cb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:09.565096 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94faa8cb-b730-4ffe-85cd-bcc44440c7cb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:09.565376 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94faa8cb-b730-4ffe-85cd-bcc44440c7cb None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:09.573960 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:09.577871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94faa8cb-b730-4ffe-85cd-bcc44440c7cb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:55:09.586683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-94faa8cb-b730-4ffe-85cd-bcc44440c7cb None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 15:55:09.586683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:09.586683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:09.586683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:09.586683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:09.586683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:09.586683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:09.586683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:09.586683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:09.586683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:09.586683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:09.586683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:09.586683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:55:09.586683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:55:09.586683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:55:09.586683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:55:09.587660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:55:09.587660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:55:09.587660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:55:09.587660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:55:09.587660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:09.587660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:09.587660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:09.587660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:09.587660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:09.587660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:09.587660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:09.587660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:09.587660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:09.587660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:09.587660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:09.587660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:09.588764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:09.588764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:09.588764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:09.588764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:09.588764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:55:09.588764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:55:09.588764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:55:09.588764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:55:09.588764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:55:09.588764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:55:09.588764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 15:55:09.588764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:09.588764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 458/932] 10.222.0.85 () {56 vars in 1120 bytes} [Wed May 25 15:55:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:09.593840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94faa8cb-b730-4ffe-85cd-bcc44440c7cb None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:09.649488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9bcab2c-6571-4f97-85dd-e908e05d8b1d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:09.657302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:09.685334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99bcc0-a6cb-4f09-80a1-d28b0f8dc8d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:09.686164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99bcc0-a6cb-4f09-80a1-d28b0f8dc8d2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:09.686553 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99bcc0-a6cb-4f09-80a1-d28b0f8dc8d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:09.686956 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a99bcc0-a6cb-4f09-80a1-d28b0f8dc8d2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:09.687881 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99bcc0-a6cb-4f09-80a1-d28b0f8dc8d2 None admin] RBAC: Adding query filter params (interface=public, service_id=0876d17514ea412d8279d535e9c03b95, region_id=RegionOne) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:09.688304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99bcc0-a6cb-4f09-80a1-d28b0f8dc8d2 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:09.689785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a99bcc0-a6cb-4f09-80a1-d28b0f8dc8d2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:09.699956 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 475/933] 10.222.0.85 () {56 vars in 1251 bytes} [Wed May 25 15:55:09 2022] GET /identity/v3/endpoints?service_id=0876d17514ea412d8279d535e9c03b95&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:09.701339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99bcc0-a6cb-4f09-80a1-d28b0f8dc8d2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:09.708941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94faa8cb-b730-4ffe-85cd-bcc44440c7cb None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:09.748127 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817eb6c-ff02-4314-91e4-f22eae0bea78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:09.749173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817eb6c-ff02-4314-91e4-f22eae0bea78 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:09.749648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817eb6c-ff02-4314-91e4-f22eae0bea78 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:09.750082 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4817eb6c-ff02-4314-91e4-f22eae0bea78 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:09.751005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817eb6c-ff02-4314-91e4-f22eae0bea78 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:09.751463 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817eb6c-ff02-4314-91e4-f22eae0bea78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:09.753269 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4817eb6c-ff02-4314-91e4-f22eae0bea78 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:09.763331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 459/934] 10.222.0.85 () {56 vars in 1115 bytes} [Wed May 25 15:55:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:09.768576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817eb6c-ff02-4314-91e4-f22eae0bea78 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:09.804476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a99bcc0-a6cb-4f09-80a1-d28b0f8dc8d2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:09.839406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:09.845151 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b68b0f-f406-487f-8921-6d940d3533ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:09.846027 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b68b0f-f406-487f-8921-6d940d3533ef None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:09.846465 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b68b0f-f406-487f-8921-6d940d3533ef None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:09.846896 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14b68b0f-f406-487f-8921-6d940d3533ef None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:09.847793 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b68b0f-f406-487f-8921-6d940d3533ef None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:09.850212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14b68b0f-f406-487f-8921-6d940d3533ef None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:09.857216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 476/935] 10.222.0.85 () {56 vars in 1087 bytes} [Wed May 25 15:55:09 2022] GET /identity/v3/services => generated 1497 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 15:55:09.885332 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4817eb6c-ff02-4314-91e4-f22eae0bea78 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:09.923483 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c0f37-83f2-4fe5-9a64-bc30304ff95f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:09.924374 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c0f37-83f2-4fe5-9a64-bc30304ff95f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:09.924658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c0f37-83f2-4fe5-9a64-bc30304ff95f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:09.924939 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152c0f37-83f2-4fe5-9a64-bc30304ff95f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:09.937463 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c0f37-83f2-4fe5-9a64-bc30304ff95f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:09.939011 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-152c0f37-83f2-4fe5-9a64-bc30304ff95f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:09.958737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-152c0f37-83f2-4fe5-9a64-bc30304ff95f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:09.958737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:09.958737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:09.958737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:09.958737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:09.958737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:09.958737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:09.958737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:09.958737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:09.958737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:09.958737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:09.958737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:09.958737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:55:09.958737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:55:09.958737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:55:09.959746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:55:09.959746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:09.959746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:09.959746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:09.959746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:09.959746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:09.959746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:09.959746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:09.959746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:09.959746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:09.959746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:09.959746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:09.959746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:09.959746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:09.959746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:09.959746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:09.959746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:09.960762 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:55:09.960762 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:55:09.960762 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:09.960762 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:09.960762 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 460/936] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:55:09.967522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b68b0f-f406-487f-8921-6d940d3533ef None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:10.009102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:10.081162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14b68b0f-f406-487f-8921-6d940d3533ef None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:10.116158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1cecb-d9d1-4f7f-a169-1a2278018e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:10.117278 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1cecb-d9d1-4f7f-a169-1a2278018e47 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:10.117782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1cecb-d9d1-4f7f-a169-1a2278018e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:10.118284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e1cecb-d9d1-4f7f-a169-1a2278018e47 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:10.119240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1cecb-d9d1-4f7f-a169-1a2278018e47 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:10.119788 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1cecb-d9d1-4f7f-a169-1a2278018e47 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:10.121478 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e1cecb-d9d1-4f7f-a169-1a2278018e47 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:10.135792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 477/937] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:55:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:10.142019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c0f37-83f2-4fe5-9a64-bc30304ff95f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:10.249675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152c0f37-83f2-4fe5-9a64-bc30304ff95f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:10.263680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:10.283903 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6650ec-8b24-4f4d-973e-9e0c5b12554b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:10.284738 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6650ec-8b24-4f4d-973e-9e0c5b12554b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:10.285030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6650ec-8b24-4f4d-973e-9e0c5b12554b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:10.285309 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6650ec-8b24-4f4d-973e-9e0c5b12554b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:10.293459 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6650ec-8b24-4f4d-973e-9e0c5b12554b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:10.294936 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6650ec-8b24-4f4d-973e-9e0c5b12554b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:10.310223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-6e6650ec-8b24-4f4d-973e-9e0c5b12554b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 15:55:10.310223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:10.310223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:10.310223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:10.310223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:10.310223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:10.310223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:10.310223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:10.310223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:10.310223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:10.310223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:10.310223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:10.310223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:55:10.310223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:55:10.310223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:55:10.310223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:10.311131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:10.312097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:10.312097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:10.312097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:55:10.312097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:55:10.312097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:55:10.312097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:55:10.312097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:55:10.312097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:55:10.312097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 15:55:10.312097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:10.312097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 461/938] 10.222.0.85 () {56 vars in 1117 bytes} [Wed May 25 15:55:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:10.318547 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1cecb-d9d1-4f7f-a169-1a2278018e47 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:10.341928 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a2e76-2b46-4284-9915-4d9d8f403fbf None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:10.342227 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a2e76-2b46-4284-9915-4d9d8f403fbf None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:10.342505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705a2e76-2b46-4284-9915-4d9d8f403fbf None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:10.347717 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 462/939] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:55:10.349331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b753e-98da-41a5-993b-164a1c224915 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:10.349624 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b753e-98da-41a5-993b-164a1c224915 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:10.349905 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b753e-98da-41a5-993b-164a1c224915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:10.367307 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4a5b753e-98da-41a5-993b-164a1c224915 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:10.368555 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4a5b753e-98da-41a5-993b-164a1c224915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:10.434542 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:10.444262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e1cecb-d9d1-4f7f-a169-1a2278018e47 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:10.448027 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 463/940] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:10 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:55:10.465593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b753e-98da-41a5-993b-164a1c224915 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:10.485019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63c002-8a07-40b3-8579-d1a98d2580c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:10.486412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63c002-8a07-40b3-8579-d1a98d2580c3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:10.487002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63c002-8a07-40b3-8579-d1a98d2580c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:10.487480 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f63c002-8a07-40b3-8579-d1a98d2580c3 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:10.488655 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63c002-8a07-40b3-8579-d1a98d2580c3 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:10.489174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63c002-8a07-40b3-8579-d1a98d2580c3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:10.491113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f63c002-8a07-40b3-8579-d1a98d2580c3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:10.510964 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 478/941] 10.222.0.85 () {56 vars in 1149 bytes} [Wed May 25 15:55:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:10.518536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63c002-8a07-40b3-8579-d1a98d2580c3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:10.566221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b753e-98da-41a5-993b-164a1c224915 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:10.568598 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:10.601765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee3d3d-00d7-4e79-b6b0-d98cae309f3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:10.602448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee3d3d-00d7-4e79-b6b0-d98cae309f3a None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:10.602739 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee3d3d-00d7-4e79-b6b0-d98cae309f3a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:10.603022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fee3d3d-00d7-4e79-b6b0-d98cae309f3a None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:10.603742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee3d3d-00d7-4e79-b6b0-d98cae309f3a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:10.605336 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fee3d3d-00d7-4e79-b6b0-d98cae309f3a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:10.606736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: WARNING keystone.server.flask.common [None req-6fee3d3d-00d7-4e79-b6b0-d98cae309f3a None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. May 25 15:55:10.626029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.sql.core [None req-6fee3d3d-00d7-4e79-b6b0-d98cae309f3a None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 15:55:10.626029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 25 15:55:10.626029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [parameters: {'id': '43d1c6f502a94d6581abeb035be2f315', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 25 15:55:10.626029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80232) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 15:55:10.631706 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f63c002-8a07-40b3-8579-d1a98d2580c3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:10.642973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-6fee3d3d-00d7-4e79-b6b0-d98cae309f3a None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 43d1c6f502a94d6581abeb035be2f315.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 43d1c6f502a94d6581abeb035be2f315. May 25 15:55:10.642973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:10.642973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 15:55:10.642973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 15:55:10.642973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 15:55:10.642973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 15:55:10.642973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 15:55:10.642973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result = self._query(query) May 25 15:55:10.642973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 15:55:10.642973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application conn.query(q) May 25 15:55:10.642973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 15:55:10.642973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 15:55:10.642973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result.read() May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project May 25 15:55:10.643865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return project_ref.to_dict() May 25 15:55:10.644847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 15:55:10.644847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application next(self.gen) May 25 15:55:10.644847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 15:55:10.644847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application yield resource May 25 15:55:10.644847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 15:55:10.644847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application next(self.gen) May 25 15:55:10.644847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 15:55:10.644847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.session.rollback() May 25 15:55:10.644847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 15:55:10.644847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.force_reraise() May 25 15:55:10.644847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 15:55:10.644847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise self.value May 25 15:55:10.644847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 15:55:10.644847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 15:55:10.644847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 15:55:10.644847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application session.commit() May 25 15:55:10.644847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 15:55:10.645770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 15:55:10.645770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 15:55:10.645770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._prepare_impl() May 25 15:55:10.645770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 15:55:10.645770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.session.flush() May 25 15:55:10.645770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 15:55:10.645770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._flush(objects) May 25 15:55:10.645770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 15:55:10.645770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 15:55:10.645770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 15:55:10.645770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application compat.raise_( May 25 15:55:10.645770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 15:55:10.645770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception May 25 15:55:10.645770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 15:55:10.645770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application flush_context.execute() May 25 15:55:10.645770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 15:55:10.645770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rec.execute(self) May 25 15:55:10.647356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 15:55:10.647356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 15:55:10.647356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 15:55:10.647356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 15:55:10.647356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements May 25 15:55:10.647356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application c = connection._execute_20( May 25 15:55:10.647356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 15:55:10.647356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 15:55:10.647356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 15:55:10.647356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 15:55:10.647356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 15:55:10.647356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 15:55:10.647356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 15:55:10.647356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 15:55:10.647356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 15:55:10.648197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 15:55:10.648197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 15:55:10.648197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception May 25 15:55:10.648197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 15:55:10.648197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 15:55:10.648197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 15:55:10.648197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 15:55:10.648197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 15:55:10.648197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result = self._query(query) May 25 15:55:10.648197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 15:55:10.648197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application conn.query(q) May 25 15:55:10.648197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 15:55:10.648197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 15:55:10.648197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 15:55:10.648197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result.read() May 25 15:55:10.648197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 15:55:10.648197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 15:55:10.649101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 15:55:10.649101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 15:55:10.649101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 15:55:10.649101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 15:55:10.649101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 15:55:10.649101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 15:55:10.649101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") May 25 15:55:10.649101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] May 25 15:55:10.649101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [parameters: {'id': '43d1c6f502a94d6581abeb035be2f315', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] May 25 15:55:10.649101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 15:55:10.649101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:10.649101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:10.649101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:10.649101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:10.649101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project May 25 15:55:10.649883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) May 25 15:55:10.649883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 15:55:10.649883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 15:55:10.649883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. May 25 15:55:10.649883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:10.649883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:10.649883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:10.649883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:10.649883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:10.649883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:10.649883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:10.649883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:10.649883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:10.649883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:10.649883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:10.649883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:10.649883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:10.652254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:10.652254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post May 25 15:55:10.652254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( May 25 15:55:10.652254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:10.652254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:10.652254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project May 25 15:55:10.652254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.Conflict( May 25 15:55:10.652254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 43d1c6f502a94d6581abeb035be2f315. May 25 15:55:10.652254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:10.652254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 464/942] 10.222.0.85 () {60 vars in 1140 bytes} [Wed May 25 15:55:10 2022] POST /identity/v3/projects => generated 262 bytes in 180 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) May 25 15:55:10.652254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee3d3d-00d7-4e79-b6b0-d98cae309f3a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:10.685815 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e9eb3-ba81-481a-91a9-5b41d1033c67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:10.686681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e9eb3-ba81-481a-91a9-5b41d1033c67 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:10.686971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e9eb3-ba81-481a-91a9-5b41d1033c67 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:10.687301 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811e9eb3-ba81-481a-91a9-5b41d1033c67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:10.699214 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e9eb3-ba81-481a-91a9-5b41d1033c67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:10.700252 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811e9eb3-ba81-481a-91a9-5b41d1033c67 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:10.719814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-811e9eb3-ba81-481a-91a9-5b41d1033c67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:10.719814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:10.719814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:10.719814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:10.719814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:10.719814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:10.719814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:10.719814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:10.719814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:10.719814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:10.719814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:10.719814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:10.719814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:55:10.719814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:55:10.719814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:55:10.720782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:55:10.720782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:10.720782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:10.720782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:10.720782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:10.720782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:10.720782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:10.720782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:10.720782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:10.720782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:10.720782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:10.720782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:10.720782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:10.720782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:10.720782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:10.720782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:10.720782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:10.721689 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:55:10.721689 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:55:10.721689 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:10.721689 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:10.721689 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 479/943] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:10.726512 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e9eb3-ba81-481a-91a9-5b41d1033c67 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:10.731299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:10.776126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fee3d3d-00d7-4e79-b6b0-d98cae309f3a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:10.810390 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb84530-258d-4e51-b0d4-f4aa1c065836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:10.811261 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb84530-258d-4e51-b0d4-f4aa1c065836 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:10.811567 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb84530-258d-4e51-b0d4-f4aa1c065836 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:10.811865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb84530-258d-4e51-b0d4-f4aa1c065836 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:10.818526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb84530-258d-4e51-b0d4-f4aa1c065836 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:10.820339 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb84530-258d-4e51-b0d4-f4aa1c065836 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:10.834676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-1eb84530-258d-4e51-b0d4-f4aa1c065836 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:10.834676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:10.834676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:10.834676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:10.834676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:10.834676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:10.834676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:10.834676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:10.834676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:10.834676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:10.834676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:10.834676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:10.834676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:55:10.834676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:55:10.834676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:55:10.835586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:55:10.835586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:10.835586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:10.835586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:10.835586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:10.835586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:10.835586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:10.835586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:10.835586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:10.835586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:10.835586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:10.835586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:10.835586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:10.835586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:10.835586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:10.835586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:10.835586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:10.836530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:55:10.836530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:55:10.836530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:55:10.836530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:55:10.836530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:55:10.836530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:55:10.836530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:10.836530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:10.836530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 465/944] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:55:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:55:10.843862 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb84530-258d-4e51-b0d4-f4aa1c065836 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:10.859812 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811e9eb3-ba81-481a-91a9-5b41d1033c67 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:10.869600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:10.897228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a6478-72fd-4add-818d-49efa71f1437 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:10.898116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a6478-72fd-4add-818d-49efa71f1437 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:10.898314 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a6478-72fd-4add-818d-49efa71f1437 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:10.898556 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a6478-72fd-4add-818d-49efa71f1437 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:10.899290 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a6478-72fd-4add-818d-49efa71f1437 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:10.899603 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a6478-72fd-4add-818d-49efa71f1437 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:10.901038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487a6478-72fd-4add-818d-49efa71f1437 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:10.918342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 480/945] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:55:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:10.919213 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a6478-72fd-4add-818d-49efa71f1437 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:10.957346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb84530-258d-4e51-b0d4-f4aa1c065836 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:10.984354 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:10.996388 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52c384-51b3-4bbc-864a-b7f0362c5887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:10.997537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52c384-51b3-4bbc-864a-b7f0362c5887 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:10.998066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52c384-51b3-4bbc-864a-b7f0362c5887 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:10.998549 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e52c384-51b3-4bbc-864a-b7f0362c5887 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:10.999664 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52c384-51b3-4bbc-864a-b7f0362c5887 None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:11.000237 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52c384-51b3-4bbc-864a-b7f0362c5887 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:11.001958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e52c384-51b3-4bbc-864a-b7f0362c5887 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:11.031217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a6478-72fd-4add-818d-49efa71f1437 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:11.031785 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 466/946] 10.222.0.85 () {56 vars in 1112 bytes} [Wed May 25 15:55:10 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:11.066831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881dae2-166d-48f6-bbcc-2da43755a7c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:11.067751 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881dae2-166d-48f6-bbcc-2da43755a7c3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:11.068105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881dae2-166d-48f6-bbcc-2da43755a7c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:11.068392 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3881dae2-166d-48f6-bbcc-2da43755a7c3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:11.076616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881dae2-166d-48f6-bbcc-2da43755a7c3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:11.078113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3881dae2-166d-48f6-bbcc-2da43755a7c3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:11.094286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3881dae2-166d-48f6-bbcc-2da43755a7c3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:11.094286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:11.094286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:11.094286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:11.094286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:11.094286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:11.094286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:11.094286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:11.094286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:11.094286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:11.094286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:11.094286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:11.094286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:55:11.094286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:55:11.094286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:55:11.095592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:55:11.095592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:11.095592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:11.095592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:11.095592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:11.095592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:11.095592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:11.095592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:11.095592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:11.095592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:11.095592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:11.095592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:11.095592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:11.095592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:11.095592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:11.095592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:11.095592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:11.097196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:55:11.097196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:55:11.097196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:55:11.097196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:55:11.097196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:55:11.097196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:55:11.097196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:11.097196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:11.099959 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 481/947] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:55:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:55:11.101873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52c384-51b3-4bbc-864a-b7f0362c5887 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:11.196572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:11.219358 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e52c384-51b3-4bbc-864a-b7f0362c5887 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:11.255631 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bf5aa77-3fa3-4561-895a-6f9b9c307fa9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:11.256345 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5aa77-3fa3-4561-895a-6f9b9c307fa9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:11.256639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5aa77-3fa3-4561-895a-6f9b9c307fa9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:11.256920 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bf5aa77-3fa3-4561-895a-6f9b9c307fa9 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:11.257716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5aa77-3fa3-4561-895a-6f9b9c307fa9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:11.258035 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5aa77-3fa3-4561-895a-6f9b9c307fa9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:11.259490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bf5aa77-3fa3-4561-895a-6f9b9c307fa9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:11.272469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 467/948] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:55:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:11.278366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881dae2-166d-48f6-bbcc-2da43755a7c3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:11.389417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3881dae2-166d-48f6-bbcc-2da43755a7c3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:11.417945 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:11.430807 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4c78227-9846-411f-b149-8e1d899b15fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:11.431723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c78227-9846-411f-b149-8e1d899b15fe None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:11.432054 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c78227-9846-411f-b149-8e1d899b15fe None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:11.432402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4c78227-9846-411f-b149-8e1d899b15fe None admin] PATH_INFO: `/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/066de1f11f724eee9df0377beb58d92c/roles/f1cd5d80f39b48bdb388dd5c539d9a0b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:11.434706 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c78227-9846-411f-b149-8e1d899b15fe None admin] RBAC: Authorizing `identity:create_grant(project_id=5584ebdaf133444ca69efc5554bf48e9, user_id=066de1f11f724eee9df0377beb58d92c, role_id=f1cd5d80f39b48bdb388dd5c539d9a0b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:11.436205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4c78227-9846-411f-b149-8e1d899b15fe None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:11.454566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 482/949] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:55:11 2022] PUT /identity/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/066de1f11f724eee9df0377beb58d92c/roles/f1cd5d80f39b48bdb388dd5c539d9a0b => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 15:55:11.612371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacda44f-11ee-4ee5-a4bc-e18c428a1939 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:11.612890 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacda44f-11ee-4ee5-a4bc-e18c428a1939 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:11.613126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cacda44f-11ee-4ee5-a4bc-e18c428a1939 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:11.615009 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 468/950] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:11.621218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6863c8-013f-49c9-89f8-cadd890d6bc0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:11.621761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6863c8-013f-49c9-89f8-cadd890d6bc0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:11.622017 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6863c8-013f-49c9-89f8-cadd890d6bc0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:11.641150 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bb6863c8-013f-49c9-89f8-cadd890d6bc0 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:11.642296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bb6863c8-013f-49c9-89f8-cadd890d6bc0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:11.718581 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 483/951] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:11.738326 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacda44f-11ee-4ee5-a4bc-e18c428a1939 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:11.844929 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cacda44f-11ee-4ee5-a4bc-e18c428a1939 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:11.880686 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8548cc-a713-4b60-b50c-60d0957e944d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:11.881659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8548cc-a713-4b60-b50c-60d0957e944d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:11.882046 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8548cc-a713-4b60-b50c-60d0957e944d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:11.882426 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8548cc-a713-4b60-b50c-60d0957e944d None admin] PATH_INFO: `/v3/services/compute` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:11.883222 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8548cc-a713-4b60-b50c-60d0957e944d None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:11.884733 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf8548cc-a713-4b60-b50c-60d0957e944d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:11.897570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-cf8548cc-a713-4b60-b50c-60d0957e944d None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 15:55:11.897570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:11.897570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:55:11.897570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:55:11.897570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:55:11.897570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:55:11.897570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:55:11.897570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:11.897570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 15:55:11.897570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:11.897570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:11.897570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:11.897570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:11.897570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:11.897570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:11.897570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:11.897570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:11.898833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:11.898833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:11.898833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:11.898833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:11.898833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:11.898833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:11.898833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:55:11.898833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:55:11.898833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:55:11.898833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:55:11.898833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:11.898833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:11.898833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:11.898833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:11.898833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:11.898833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:11.899970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:11.899970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:11.899970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:11.899970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:11.899970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:11.899970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:11.899970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:11.899970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:11.899970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:11.899970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:11.899970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:55:11.899970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:11.899970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 15:55:11.899970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:11.905406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 469/952] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:55:11 2022] GET /identity/v3/services/compute => generated 88 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:11.908633 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6863c8-013f-49c9-89f8-cadd890d6bc0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:12.018037 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6863c8-013f-49c9-89f8-cadd890d6bc0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:12.036504 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:12.054388 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c823db2-a2f6-494e-8f88-e73629b97f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:12.055220 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c823db2-a2f6-494e-8f88-e73629b97f1e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:12.055511 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c823db2-a2f6-494e-8f88-e73629b97f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:12.055794 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c823db2-a2f6-494e-8f88-e73629b97f1e None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:12.056546 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c823db2-a2f6-494e-8f88-e73629b97f1e None admin] RBAC: Adding query filter params (name=compute) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:12.056867 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c823db2-a2f6-494e-8f88-e73629b97f1e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:12.058340 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c823db2-a2f6-494e-8f88-e73629b97f1e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:12.067944 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 484/953] 10.222.0.85 () {56 vars in 1112 bytes} [Wed May 25 15:55:11 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:12.069616 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8548cc-a713-4b60-b50c-60d0957e944d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:12.084187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:12.114539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98158bc-c613-439e-90e6-7d68779bfedb None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:12.114832 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98158bc-c613-439e-90e6-7d68779bfedb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:12.115102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98158bc-c613-439e-90e6-7d68779bfedb None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:12.117228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 485/954] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:12.124111 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19b005-65d8-45b0-8a41-2cb55b24e794 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:12.124521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19b005-65d8-45b0-8a41-2cb55b24e794 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:12.125072 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a19b005-65d8-45b0-8a41-2cb55b24e794 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:12.147511 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9a19b005-65d8-45b0-8a41-2cb55b24e794 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:12.148894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9a19b005-65d8-45b0-8a41-2cb55b24e794 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:12.185500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8548cc-a713-4b60-b50c-60d0957e944d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:12.223575 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130fbc3-4837-467b-87ca-a0a99ea9069d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:12.224655 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130fbc3-4837-467b-87ca-a0a99ea9069d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:12.225117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130fbc3-4837-467b-87ca-a0a99ea9069d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:12.225567 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4130fbc3-4837-467b-87ca-a0a99ea9069d None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:12.226583 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130fbc3-4837-467b-87ca-a0a99ea9069d None admin] RBAC: Adding query filter params (type=compute) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:12.227071 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130fbc3-4837-467b-87ca-a0a99ea9069d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:12.228685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4130fbc3-4837-467b-87ca-a0a99ea9069d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:12.239861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 470/955] 10.222.0.85 () {56 vars in 1112 bytes} [Wed May 25 15:55:12 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:12.251984 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 486/956] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:12 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:12.267903 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130fbc3-4837-467b-87ca-a0a99ea9069d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:12.347918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:12.385339 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4130fbc3-4837-467b-87ca-a0a99ea9069d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:12.423261 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5232071-a76e-411b-b165-600d9cf2a58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:12.424494 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5232071-a76e-411b-b165-600d9cf2a58a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:12.425014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5232071-a76e-411b-b165-600d9cf2a58a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:12.425496 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5232071-a76e-411b-b165-600d9cf2a58a None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:12.426366 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5232071-a76e-411b-b165-600d9cf2a58a None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:12.428030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5232071-a76e-411b-b165-600d9cf2a58a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:12.441672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-e5232071-a76e-411b-b165-600d9cf2a58a None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 15:55:12.441672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:12.441672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:55:12.441672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:55:12.441672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:55:12.441672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:55:12.441672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:55:12.441672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:12.441672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 15:55:12.441672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:12.441672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:12.441672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:12.441672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:12.441672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:12.441672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:12.441672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:12.441672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:12.442780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:12.442780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:12.442780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:12.442780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:12.442780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:12.442780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:12.442780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:55:12.442780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:55:12.442780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:55:12.442780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:55:12.442780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:12.442780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:12.442780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:12.442780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:12.442780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:12.442780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:12.443635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:12.443635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:12.443635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:12.443635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:12.443635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:12.443635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:12.443635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:12.443635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:12.443635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:12.443635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:12.443635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:55:12.443635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:12.443635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 15:55:12.443635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:12.446311 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 471/957] 10.222.0.85 () {56 vars in 1126 bytes} [Wed May 25 15:55:12 2022] GET /identity/v3/services/object-store => generated 93 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:12.453767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19b005-65d8-45b0-8a41-2cb55b24e794 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:12.558539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a19b005-65d8-45b0-8a41-2cb55b24e794 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:12.568426 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:12.595056 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3270c-a959-4d6e-ac56-b72c2d6b608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:12.595737 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3270c-a959-4d6e-ac56-b72c2d6b608a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:12.596054 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3270c-a959-4d6e-ac56-b72c2d6b608a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:12.596339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20f3270c-a959-4d6e-ac56-b72c2d6b608a None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:12.597071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3270c-a959-4d6e-ac56-b72c2d6b608a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:12.597391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3270c-a959-4d6e-ac56-b72c2d6b608a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:12.598830 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20f3270c-a959-4d6e-ac56-b72c2d6b608a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:12.605162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 487/958] 10.222.0.85 () {56 vars in 1122 bytes} [Wed May 25 15:55:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:12.611059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5232071-a76e-411b-b165-600d9cf2a58a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:12.714064 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5232071-a76e-411b-b165-600d9cf2a58a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:12.748193 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:12.750223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f02698-3814-43ee-859a-8869fe38c25f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:12.750873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f02698-3814-43ee-859a-8869fe38c25f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:12.751163 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f02698-3814-43ee-859a-8869fe38c25f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:12.751451 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f02698-3814-43ee-859a-8869fe38c25f None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:12.752215 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f02698-3814-43ee-859a-8869fe38c25f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:12.752525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f02698-3814-43ee-859a-8869fe38c25f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:12.753909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f02698-3814-43ee-859a-8869fe38c25f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:12.759445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 472/959] 10.222.0.85 () {56 vars in 1122 bytes} [Wed May 25 15:55:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:12.766195 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3270c-a959-4d6e-ac56-b72c2d6b608a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:12.877071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20f3270c-a959-4d6e-ac56-b72c2d6b608a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:12.912535 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67eb5618-8817-4082-bddf-9db9fb2be69d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:12.913208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb5618-8817-4082-bddf-9db9fb2be69d None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:12.913500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb5618-8817-4082-bddf-9db9fb2be69d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:12.913787 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67eb5618-8817-4082-bddf-9db9fb2be69d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:12.914549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb5618-8817-4082-bddf-9db9fb2be69d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:12.916054 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67eb5618-8817-4082-bddf-9db9fb2be69d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:12.941999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 488/960] 10.222.0.85 () {60 vars in 1144 bytes} [Wed May 25 15:55:12 2022] POST /identity/v3/endpoints => generated 349 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:55:13.423649 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113bf1e-e9e7-432c-9286-cbddf81532fc None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:13.424090 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113bf1e-e9e7-432c-9286-cbddf81532fc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:13.424268 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7113bf1e-e9e7-432c-9286-cbddf81532fc None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:13.426145 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 473/961] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:13.432098 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdabf44-e2f8-4fb3-819a-37400dab2801 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:13.432493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdabf44-e2f8-4fb3-819a-37400dab2801 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:13.432739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdabf44-e2f8-4fb3-819a-37400dab2801 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:13.454801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-cfdabf44-e2f8-4fb3-819a-37400dab2801 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:13.456366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cfdabf44-e2f8-4fb3-819a-37400dab2801 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:13.480142 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:13.535745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 489/962] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:13.560737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113bf1e-e9e7-432c-9286-cbddf81532fc None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:13.666200 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7113bf1e-e9e7-432c-9286-cbddf81532fc None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:13.701486 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d55b4-5d47-47e4-ba06-557aa878aa83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:13.702544 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d55b4-5d47-47e4-ba06-557aa878aa83 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:13.703034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d55b4-5d47-47e4-ba06-557aa878aa83 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:13.703540 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268d55b4-5d47-47e4-ba06-557aa878aa83 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:13.704604 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d55b4-5d47-47e4-ba06-557aa878aa83 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:13.706630 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268d55b4-5d47-47e4-ba06-557aa878aa83 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:13.706842 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907826d-41e2-4ad9-8f3e-12507a4f1c4e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:13.707139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907826d-41e2-4ad9-8f3e-12507a4f1c4e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:13.707425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b907826d-41e2-4ad9-8f3e-12507a4f1c4e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:13.709095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 490/963] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:13.714214 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b916b0-8c06-4da1-9d42-4187620d1623 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:13.714514 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b916b0-8c06-4da1-9d42-4187620d1623 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:13.714805 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b916b0-8c06-4da1-9d42-4187620d1623 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:13.722856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.sql.core [None req-268d55b4-5d47-47e4-ba06-557aa878aa83 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 15:55:13.722856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 25 15:55:13.722856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [parameters: {'user_id': '90bf539b4e85492ea7795d3a078fd934', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 25 15:55:13.722856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80232) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} May 25 15:55:13.731435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-268d55b4-5d47-47e4-ba06-557aa878aa83 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 25 15:55:13.731435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:13.731435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 15:55:13.731435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 15:55:13.731435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 15:55:13.731435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 15:55:13.731435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 15:55:13.731435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result = self._query(query) May 25 15:55:13.731435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 15:55:13.731435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application conn.query(q) May 25 15:55:13.731435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 15:55:13.731435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 15:55:13.731435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 15:55:13.731435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result.read() May 25 15:55:13.731435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return method(*args, **kwargs) May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 15:55:13.732359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application next(self.gen) May 25 15:55:13.733310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope May 25 15:55:13.733310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application yield resource May 25 15:55:13.733310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ May 25 15:55:13.733310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application next(self.gen) May 25 15:55:13.733310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session May 25 15:55:13.733310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.session.rollback() May 25 15:55:13.733310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ May 25 15:55:13.733310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.force_reraise() May 25 15:55:13.733310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise May 25 15:55:13.733310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise self.value May 25 15:55:13.733310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session May 25 15:55:13.733310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) May 25 15:55:13.733310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction May 25 15:55:13.733310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application session.commit() May 25 15:55:13.733310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1435, in commit May 25 15:55:13.733310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) May 25 15:55:13.733310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit May 25 15:55:13.734220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._prepare_impl() May 25 15:55:13.734220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl May 25 15:55:13.734220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.session.flush() May 25 15:55:13.734220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3367, in flush May 25 15:55:13.734220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._flush(objects) May 25 15:55:13.734220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3507, in _flush May 25 15:55:13.734220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) May 25 15:55:13.734220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ May 25 15:55:13.734220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application compat.raise_( May 25 15:55:13.734220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 15:55:13.734220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception May 25 15:55:13.734220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3467, in _flush May 25 15:55:13.734220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application flush_context.execute() May 25 15:55:13.734220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute May 25 15:55:13.734220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rec.execute(self) May 25 15:55:13.734220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute May 25 15:55:13.734220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( May 25 15:55:13.735194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj May 25 15:55:13.735194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application _emit_insert_statements( May 25 15:55:13.735194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements May 25 15:55:13.735194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result = connection._execute_20( May 25 15:55:13.735194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 May 25 15:55:13.735194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) May 25 15:55:13.735194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection May 25 15:55:13.735194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return connection._execute_clauseelement( May 25 15:55:13.735194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement May 25 15:55:13.735194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ret = self._execute_context( May 25 15:55:13.735194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context May 25 15:55:13.735194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._handle_dbapi_exception( May 25 15:55:13.735194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception May 25 15:55:13.735194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) May 25 15:55:13.735194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 207, in raise_ May 25 15:55:13.735194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception May 25 15:55:13.736107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context May 25 15:55:13.736107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self.dialect.do_execute( May 25 15:55:13.736107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute May 25 15:55:13.736107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) May 25 15:55:13.736107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute May 25 15:55:13.736107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result = self._query(query) May 25 15:55:13.736107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query May 25 15:55:13.736107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application conn.query(q) May 25 15:55:13.736107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query May 25 15:55:13.736107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) May 25 15:55:13.736107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result May 25 15:55:13.736107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application result.read() May 25 15:55:13.736107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read May 25 15:55:13.736107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() May 25 15:55:13.736107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet May 25 15:55:13.736107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application packet.raise_for_error() May 25 15:55:13.736951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error May 25 15:55:13.736951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) May 25 15:55:13.736951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception May 25 15:55:13.736951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise errorclass(errno, errval) May 25 15:55:13.736951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") May 25 15:55:13.736951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] May 25 15:55:13.736951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [parameters: {'user_id': '90bf539b4e85492ea7795d3a078fd934', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] May 25 15:55:13.736951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) May 25 15:55:13.736951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:13.736951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:13.736951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:13.736951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:13.736951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:13.736951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:13.736951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:13.738144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:13.738144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:13.738144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:13.738144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:13.738144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:13.738144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:13.738144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:13.738144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post May 25 15:55:13.738144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( May 25 15:55:13.738144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:13.738144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:13.738144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:55:13.738144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:13.738144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:55:13.738144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:13.738144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user May 25 15:55:13.738144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) May 25 15:55:13.739143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects May 25 15:55:13.739143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) May 25 15:55:13.739143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper May 25 15:55:13.739143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, May 25 15:55:13.739143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. May 25 15:55:13.739143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:13.739571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-80b916b0-8c06-4da1-9d42-4187620d1623 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:13.739659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 474/964] 10.222.0.85 () {60 vars in 1131 bytes} [Wed May 25 15:55:13 2022] POST /identity/v3/users => generated 160 bytes in 173 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:13.739740 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-80b916b0-8c06-4da1-9d42-4187620d1623 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:13.739851 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d55b4-5d47-47e4-ba06-557aa878aa83 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:13.817340 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 491/965] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:13 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:55:13.824299 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:13.842943 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b916b0-8c06-4da1-9d42-4187620d1623 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:13.874765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268d55b4-5d47-47e4-ba06-557aa878aa83 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:13.913061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aa77e-c66f-462c-b826-28f7f2de4d09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:13.914813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aa77e-c66f-462c-b826-28f7f2de4d09 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:13.915552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aa77e-c66f-462c-b826-28f7f2de4d09 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:13.916082 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559aa77e-c66f-462c-b826-28f7f2de4d09 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:13.933184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559aa77e-c66f-462c-b826-28f7f2de4d09 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:13.935636 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559aa77e-c66f-462c-b826-28f7f2de4d09 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:13.945519 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:13.962447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-559aa77e-c66f-462c-b826-28f7f2de4d09 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. May 25 15:55:13.962447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:13.962447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:13.962447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:13.962447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:13.962447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:13.962447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:13.962447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:13.962447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:13.962447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:13.962447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:13.962447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:13.962447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:55:13.962447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:55:13.962447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:55:13.962447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:13.963465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:13.964480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:13.964480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:13.964480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:55:13.964480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:55:13.964480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:55:13.964480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:55:13.964480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:55:13.964480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:55:13.964480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. May 25 15:55:13.964480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:13.965044 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b916b0-8c06-4da1-9d42-4187620d1623 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:13.965753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 475/966] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:55:13 2022] GET /identity/v3/users/cinder => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:55:13.973436 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aa77e-c66f-462c-b826-28f7f2de4d09 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:14.004377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf87ba-c18a-47f7-b8bb-3deafd99e0f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:14.005213 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf87ba-c18a-47f7-b8bb-3deafd99e0f1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:14.005505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf87ba-c18a-47f7-b8bb-3deafd99e0f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:14.005788 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0bf87ba-c18a-47f7-b8bb-3deafd99e0f1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:14.019877 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0bf87ba-c18a-47f7-b8bb-3deafd99e0f1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:55:14.030425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a0bf87ba-c18a-47f7-b8bb-3deafd99e0f1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 15:55:14.030425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:14.030425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:14.030425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:14.030425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:14.030425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:14.030425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:14.030425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:14.030425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:14.030425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:14.030425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:14.030425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:14.030425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:55:14.030425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:55:14.030425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:55:14.030425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:55:14.031404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:55:14.031404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:55:14.031404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:55:14.031404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:55:14.031404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:14.031404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:14.031404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:14.031404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:14.031404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:14.031404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:14.031404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:14.031404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:14.031404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:14.031404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:14.031404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:14.031404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:14.032665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:14.032665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:14.032665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:14.032665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:14.032665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:55:14.032665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:55:14.032665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:55:14.032665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:55:14.032665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:55:14.032665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:55:14.032665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 15:55:14.032665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:14.032665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 492/967] 10.222.0.85 () {56 vars in 1120 bytes} [Wed May 25 15:55:13 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:14.038731 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf87ba-c18a-47f7-b8bb-3deafd99e0f1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:14.106026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559aa77e-c66f-462c-b826-28f7f2de4d09 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:14.130487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:14.154166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e3a52-aace-4a16-9bff-ee16e9e286f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:14.155328 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e3a52-aace-4a16-9bff-ee16e9e286f6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:14.155877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e3a52-aace-4a16-9bff-ee16e9e286f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:14.156382 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4e3a52-aace-4a16-9bff-ee16e9e286f6 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:14.157504 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e3a52-aace-4a16-9bff-ee16e9e286f6 None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:14.158019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e3a52-aace-4a16-9bff-ee16e9e286f6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:14.159675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd4e3a52-aace-4a16-9bff-ee16e9e286f6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:14.169680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0bf87ba-c18a-47f7-b8bb-3deafd99e0f1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:14.202131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 476/968] 10.222.0.85 () {56 vars in 1101 bytes} [Wed May 25 15:55:13 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:14.210279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c327c47-f06c-415f-92f9-eb0abd44ec62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:14.211358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c327c47-f06c-415f-92f9-eb0abd44ec62 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:14.211916 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c327c47-f06c-415f-92f9-eb0abd44ec62 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:14.212411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c327c47-f06c-415f-92f9-eb0abd44ec62 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:14.213485 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c327c47-f06c-415f-92f9-eb0abd44ec62 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:14.214071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c327c47-f06c-415f-92f9-eb0abd44ec62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:14.215780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c327c47-f06c-415f-92f9-eb0abd44ec62 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:14.227645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 493/969] 10.222.0.85 () {56 vars in 1115 bytes} [Wed May 25 15:55:14 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:14.233377 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e3a52-aace-4a16-9bff-ee16e9e286f6 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:14.358745 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd4e3a52-aace-4a16-9bff-ee16e9e286f6 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:14.396244 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45c8a7-4519-4a2e-afb3-854ea2794ae5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:14.397071 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45c8a7-4519-4a2e-afb3-854ea2794ae5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:14.397364 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45c8a7-4519-4a2e-afb3-854ea2794ae5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:14.397647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb45c8a7-4519-4a2e-afb3-854ea2794ae5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:14.408512 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45c8a7-4519-4a2e-afb3-854ea2794ae5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:14.410043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb45c8a7-4519-4a2e-afb3-854ea2794ae5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:14.430031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-cb45c8a7-4519-4a2e-afb3-854ea2794ae5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:14.430031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:14.430031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:14.430031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:14.430031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:14.430031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:14.430031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:14.430031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:14.430031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:14.430031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:14.430031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:14.430031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:14.430031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:55:14.430031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:55:14.430031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:55:14.431049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:55:14.431049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:14.431049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:14.431049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:14.431049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:14.431049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:14.431049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:14.431049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:14.431049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:14.431049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:14.431049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:14.431049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:14.431049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:14.431049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:14.431049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:14.431049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:14.431049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:14.431979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:55:14.431979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:55:14.431979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:14.431979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:14.431979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 477/970] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:55:14.438352 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c327c47-f06c-415f-92f9-eb0abd44ec62 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:14.449278 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:14.546195 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c327c47-f06c-415f-92f9-eb0abd44ec62 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:14.579596 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecef8b3c-43d1-4986-9398-d1bd0d54f530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:14.580688 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef8b3c-43d1-4986-9398-d1bd0d54f530 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:14.581028 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef8b3c-43d1-4986-9398-d1bd0d54f530 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:14.581310 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecef8b3c-43d1-4986-9398-d1bd0d54f530 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:14.582043 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef8b3c-43d1-4986-9398-d1bd0d54f530 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:14.582355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef8b3c-43d1-4986-9398-d1bd0d54f530 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:14.583754 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecef8b3c-43d1-4986-9398-d1bd0d54f530 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:14.599933 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 494/971] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:55:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:14.602698 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45c8a7-4519-4a2e-afb3-854ea2794ae5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:14.630617 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5962f7-3434-4832-b856-8dd7adc44f0f None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:14.630973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5962f7-3434-4832-b856-8dd7adc44f0f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:14.631199 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc5962f7-3434-4832-b856-8dd7adc44f0f None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:14.632641 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 495/972] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:14.633813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:14.642876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8f97f-ee4b-4838-badd-4d6c34bc91ca None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:14.643208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8f97f-ee4b-4838-badd-4d6c34bc91ca None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:14.643498 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb8f97f-ee4b-4838-badd-4d6c34bc91ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:14.660239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6eb8f97f-ee4b-4838-badd-4d6c34bc91ca None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:14.661432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6eb8f97f-ee4b-4838-badd-4d6c34bc91ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:14.715846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb45c8a7-4519-4a2e-afb3-854ea2794ae5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:14.724901 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:14.734200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 496/973] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:14 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:14.750756 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aed9d5-1d00-481f-b7f9-0a76531ed631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:14.751589 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aed9d5-1d00-481f-b7f9-0a76531ed631 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:14.751904 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aed9d5-1d00-481f-b7f9-0a76531ed631 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:14.752187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aed9d5-1d00-481f-b7f9-0a76531ed631 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:14.755983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8f97f-ee4b-4838-badd-4d6c34bc91ca None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:14.760562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aed9d5-1d00-481f-b7f9-0a76531ed631 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:14.762036 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44aed9d5-1d00-481f-b7f9-0a76531ed631 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:14.776809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-44aed9d5-1d00-481f-b7f9-0a76531ed631 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 15:55:14.776809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:14.776809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:14.776809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:14.776809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:14.776809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:14.776809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:14.776809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:14.776809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:14.776809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:14.776809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:14.776809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:14.776809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:55:14.776809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:55:14.776809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:55:14.776809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:14.777800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:14.779704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:14.779704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:14.779704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:55:14.779704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:55:14.779704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:55:14.779704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:55:14.779704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:55:14.779704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:55:14.779704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. May 25 15:55:14.779704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:14.779704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 478/974] 10.222.0.85 () {56 vars in 1117 bytes} [Wed May 25 15:55:14 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:14.783788 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aed9d5-1d00-481f-b7f9-0a76531ed631 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:14.861479 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb8f97f-ee4b-4838-badd-4d6c34bc91ca None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:14.884444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:14.897087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b0a3c-1504-4cf6-94d0-267f004a9e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:14.897603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aed9d5-1d00-481f-b7f9-0a76531ed631 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:14.897799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b0a3c-1504-4cf6-94d0-267f004a9e44 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:14.898049 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b0a3c-1504-4cf6-94d0-267f004a9e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:14.898330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98b0a3c-1504-4cf6-94d0-267f004a9e44 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:14.899062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b0a3c-1504-4cf6-94d0-267f004a9e44 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:14.900492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d98b0a3c-1504-4cf6-94d0-267f004a9e44 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:14.917062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 497/975] 10.222.0.85 () {60 vars in 1141 bytes} [Wed May 25 15:55:14 2022] POST /identity/v3/services => generated 242 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:55:14.949981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fb35dd-cb7d-4cbd-b2c2-5c35c31220cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:14.950689 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb35dd-cb7d-4cbd-b2c2-5c35c31220cf None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:14.950975 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb35dd-cb7d-4cbd-b2c2-5c35c31220cf None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:14.951254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fb35dd-cb7d-4cbd-b2c2-5c35c31220cf None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:14.952162 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb35dd-cb7d-4cbd-b2c2-5c35c31220cf None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:14.952491 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb35dd-cb7d-4cbd-b2c2-5c35c31220cf None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:14.954002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fb35dd-cb7d-4cbd-b2c2-5c35c31220cf None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:14.974430 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 479/976] 10.222.0.85 () {56 vars in 1149 bytes} [Wed May 25 15:55:14 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:14.980302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b0a3c-1504-4cf6-94d0-267f004a9e44 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:15.048313 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:15.100540 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98b0a3c-1504-4cf6-94d0-267f004a9e44 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:15.136811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787a18f-2e21-445d-8974-fedddbe0a6d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:15.137377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787a18f-2e21-445d-8974-fedddbe0a6d6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:15.137682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787a18f-2e21-445d-8974-fedddbe0a6d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:15.137963 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2787a18f-2e21-445d-8974-fedddbe0a6d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:15.203718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84df6e-3cef-438d-859e-a8b82519ef0a None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:15.204042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84df6e-3cef-438d-859e-a8b82519ef0a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:15.204326 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae84df6e-3cef-438d-859e-a8b82519ef0a None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:15.206192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 480/977] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:55:15.210908 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eec896-0652-4138-8118-83476fa11bd9 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:15.211201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eec896-0652-4138-8118-83476fa11bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:15.211485 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38eec896-0652-4138-8118-83476fa11bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:15.213422 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787a18f-2e21-445d-8974-fedddbe0a6d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:15.215167 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2787a18f-2e21-445d-8974-fedddbe0a6d6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:15.228704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-38eec896-0652-4138-8118-83476fa11bd9 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:15.230355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-38eec896-0652-4138-8118-83476fa11bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:15.277394 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:15.307867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 481/978] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:15 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:55:15.328985 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eec896-0652-4138-8118-83476fa11bd9 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:15.332895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 498/979] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:55:14 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 15:55:15.338452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787a18f-2e21-445d-8974-fedddbe0a6d6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:15.399851 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:15.430598 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38eec896-0652-4138-8118-83476fa11bd9 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:15.442421 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2787a18f-2e21-445d-8974-fedddbe0a6d6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:15.465631 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16f6d2-fdc5-4728-aef6-75ad16d0d453 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:15.466456 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16f6d2-fdc5-4728-aef6-75ad16d0d453 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:15.466743 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16f6d2-fdc5-4728-aef6-75ad16d0d453 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:15.467022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b16f6d2-fdc5-4728-aef6-75ad16d0d453 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:15.467680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b16f6d2-fdc5-4728-aef6-75ad16d0d453 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:15.469103 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b16f6d2-fdc5-4728-aef6-75ad16d0d453 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:15.478733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2430079-b50d-4c75-8270-b31a26cb4b97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:15.479606 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2430079-b50d-4c75-8270-b31a26cb4b97 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:15.479926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2430079-b50d-4c75-8270-b31a26cb4b97 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:15.480213 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2430079-b50d-4c75-8270-b31a26cb4b97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:15.481168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-7b16f6d2-fdc5-4728-aef6-75ad16d0d453 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 15:55:15.481168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:15.481168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:55:15.481168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:55:15.481168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:55:15.481168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:55:15.481168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:55:15.481168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:15.481168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 15:55:15.481168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:15.481168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:15.481168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:15.481168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:15.481168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:15.481168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:15.481168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:15.481168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:15.482123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:15.482123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:15.482123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:15.482123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:15.482123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:15.482123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:15.482123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:55:15.482123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:55:15.482123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:55:15.482123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:55:15.482123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:15.482123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:15.482123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:15.482123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:15.482123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:15.482123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:15.482986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:15.482986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:15.482986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:15.482986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:15.482986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:15.482986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:15.482986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:15.482986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:15.482986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:15.482986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:15.482986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:55:15.482986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:15.482986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 15:55:15.482986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:15.487466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 482/980] 10.222.0.85 () {56 vars in 1126 bytes} [Wed May 25 15:55:15 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:55:15.488334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16f6d2-fdc5-4728-aef6-75ad16d0d453 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:15.493424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2430079-b50d-4c75-8270-b31a26cb4b97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:15.494933 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2430079-b50d-4c75-8270-b31a26cb4b97 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:15.513269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-f2430079-b50d-4c75-8270-b31a26cb4b97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:15.513269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:15.513269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:15.513269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:15.513269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:15.513269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:15.513269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:15.513269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:15.513269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:15.513269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:15.513269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:15.513269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:15.513269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:55:15.513269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:55:15.513269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:55:15.514272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:55:15.514272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:15.514272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:15.514272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:15.514272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:15.514272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:15.514272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:15.514272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:15.514272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:15.514272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:15.514272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:15.514272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:15.514272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:15.514272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:15.514272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:15.514272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:15.514272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:15.515185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:55:15.515185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:55:15.515185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:15.515185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:15.515185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 499/981] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:15.520119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2430079-b50d-4c75-8270-b31a26cb4b97 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:15.530908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:15.591332 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b16f6d2-fdc5-4728-aef6-75ad16d0d453 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:15.621828 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2430079-b50d-4c75-8270-b31a26cb4b97 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:15.624225 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750818dc-05ae-48a8-b149-33c679638417 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:15.624904 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750818dc-05ae-48a8-b149-33c679638417 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:15.625194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750818dc-05ae-48a8-b149-33c679638417 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:15.625477 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750818dc-05ae-48a8-b149-33c679638417 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:15.626189 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750818dc-05ae-48a8-b149-33c679638417 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:15.626504 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750818dc-05ae-48a8-b149-33c679638417 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:15.627924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750818dc-05ae-48a8-b149-33c679638417 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:15.629856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:15.635201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 483/982] 10.222.0.85 () {56 vars in 1122 bytes} [Wed May 25 15:55:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:15.640939 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750818dc-05ae-48a8-b149-33c679638417 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:15.664674 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e2e38-1dd5-4664-85bf-3276a6460461 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:15.665434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e2e38-1dd5-4664-85bf-3276a6460461 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:15.665721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e2e38-1dd5-4664-85bf-3276a6460461 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:15.666386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4e2e38-1dd5-4664-85bf-3276a6460461 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:15.667401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e2e38-1dd5-4664-85bf-3276a6460461 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:15.667719 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e2e38-1dd5-4664-85bf-3276a6460461 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:15.669885 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de4e2e38-1dd5-4664-85bf-3276a6460461 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:15.683040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 500/983] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:55:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:15.688764 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e2e38-1dd5-4664-85bf-3276a6460461 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:15.745438 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750818dc-05ae-48a8-b149-33c679638417 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:15.760821 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:15.780768 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccd5cb-df3c-4024-b934-e2b515dff28f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:15.781477 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccd5cb-df3c-4024-b934-e2b515dff28f None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:15.781883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccd5cb-df3c-4024-b934-e2b515dff28f None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:15.781988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ccd5cb-df3c-4024-b934-e2b515dff28f None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:15.782707 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ccd5cb-df3c-4024-b934-e2b515dff28f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:15.783022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ccd5cb-df3c-4024-b934-e2b515dff28f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:15.784772 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9ccd5cb-df3c-4024-b934-e2b515dff28f None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:15.794030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 484/984] 10.222.0.85 () {56 vars in 1122 bytes} [Wed May 25 15:55:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:15.795087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccd5cb-df3c-4024-b934-e2b515dff28f None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:15.801298 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4e2e38-1dd5-4664-85bf-3276a6460461 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:15.837217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9d640-44e5-4622-894e-3a2613525474 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:15.838415 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9d640-44e5-4622-894e-3a2613525474 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:15.838858 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9d640-44e5-4622-894e-3a2613525474 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:15.839139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25d9d640-44e5-4622-894e-3a2613525474 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:15.846488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9d640-44e5-4622-894e-3a2613525474 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:15.847995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25d9d640-44e5-4622-894e-3a2613525474 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:15.861492 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:15.862174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-25d9d640-44e5-4622-894e-3a2613525474 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:15.862174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:15.862174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:15.862174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:15.862174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:15.862174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:15.862174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:15.862174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:15.862174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:15.862174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:15.862174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:15.862174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:15.862174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:55:15.862174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:55:15.862174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:55:15.863377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:55:15.863377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:15.863377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:15.863377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:15.863377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:15.863377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:15.863377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:15.863377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:15.863377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:15.863377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:15.863377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:15.863377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:15.863377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:15.863377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:15.863377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:15.863377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:15.863377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:15.864442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:55:15.864442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:55:15.864442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:55:15.864442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:55:15.864442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:55:15.864442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:55:15.864442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:15.864442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:15.864442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 501/985] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:55:15 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:55:15.869179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9d640-44e5-4622-894e-3a2613525474 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:15.905633 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ccd5cb-df3c-4024-b934-e2b515dff28f None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:15.940602 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc27fd7-a188-4645-a075-146abf8b772d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:15.941200 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc27fd7-a188-4645-a075-146abf8b772d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:15.941489 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc27fd7-a188-4645-a075-146abf8b772d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:15.941774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc27fd7-a188-4645-a075-146abf8b772d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:15.942578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc27fd7-a188-4645-a075-146abf8b772d None admin] RBAC: Adding query filter params (interface=admin, service_id=0876d17514ea412d8279d535e9c03b95, region_id=RegionOne) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:15.942899 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc27fd7-a188-4645-a075-146abf8b772d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:15.944367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abc27fd7-a188-4645-a075-146abf8b772d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:15.950160 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 485/986] 10.222.0.85 () {56 vars in 1249 bytes} [Wed May 25 15:55:15 2022] GET /identity/v3/endpoints?service_id=0876d17514ea412d8279d535e9c03b95&interface=admin®ion_id=RegionOne => generated 193 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:15.955754 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc27fd7-a188-4645-a075-146abf8b772d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:15.976243 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25d9d640-44e5-4622-894e-3a2613525474 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:16.004906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:16.013790 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbb37b-a3b6-45f3-84cd-1758e2361dde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:16.014477 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbb37b-a3b6-45f3-84cd-1758e2361dde None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:16.014823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbb37b-a3b6-45f3-84cd-1758e2361dde None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:16.015115 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbb37b-a3b6-45f3-84cd-1758e2361dde None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:16.015954 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbb37b-a3b6-45f3-84cd-1758e2361dde None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:16.016274 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbb37b-a3b6-45f3-84cd-1758e2361dde None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:16.017732 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbb37b-a3b6-45f3-84cd-1758e2361dde None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:16.029672 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 502/987] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:55:15 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:16.035906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbb37b-a3b6-45f3-84cd-1758e2361dde None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:16.069006 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abc27fd7-a188-4645-a075-146abf8b772d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:16.097413 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:16.107328 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efe901ee-53a6-4028-a60a-17ca3a1fe386 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:16.108045 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe901ee-53a6-4028-a60a-17ca3a1fe386 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:16.108362 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe901ee-53a6-4028-a60a-17ca3a1fe386 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:16.108628 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe901ee-53a6-4028-a60a-17ca3a1fe386 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:16.109294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe901ee-53a6-4028-a60a-17ca3a1fe386 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:16.111291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efe901ee-53a6-4028-a60a-17ca3a1fe386 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:16.120172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 486/988] 10.222.0.85 () {56 vars in 1087 bytes} [Wed May 25 15:55:15 2022] GET /identity/v3/services => generated 1727 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:55:16.144536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbb37b-a3b6-45f3-84cd-1758e2361dde None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:16.182811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c7a537-4359-440a-a6a2-be8b7c35e3a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:16.183504 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7a537-4359-440a-a6a2-be8b7c35e3a9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:16.183804 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7a537-4359-440a-a6a2-be8b7c35e3a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:16.184097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c7a537-4359-440a-a6a2-be8b7c35e3a9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:16.184978 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7a537-4359-440a-a6a2-be8b7c35e3a9 None admin] RBAC: Adding query filter params (role.id=f1cd5d80f39b48bdb388dd5c539d9a0b, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=066de1f11f724eee9df0377beb58d92c) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:16.185298 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7a537-4359-440a-a6a2-be8b7c35e3a9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:16.186829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c7a537-4359-440a-a6a2-be8b7c35e3a9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:16.193977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 503/989] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:55:16 2022] GET /identity/v3/role_assignments?user.id=066de1f11f724eee9df0377beb58d92c&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=f1cd5d80f39b48bdb388dd5c539d9a0b => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:17.139890 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf8bda-5de0-4852-a2c2-d2c57605541a None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:17.140460 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf8bda-5de0-4852-a2c2-d2c57605541a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:17.140460 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acdf8bda-5de0-4852-a2c2-d2c57605541a None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:17.142033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 487/990] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:17.147144 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420484ae-33b8-4d08-b072-f5d0d7ea7f02 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:17.147552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420484ae-33b8-4d08-b072-f5d0d7ea7f02 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:17.147740 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420484ae-33b8-4d08-b072-f5d0d7ea7f02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:17.165276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-420484ae-33b8-4d08-b072-f5d0d7ea7f02 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:17.166555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-420484ae-33b8-4d08-b072-f5d0d7ea7f02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:17.216802 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:17.239915 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 504/991] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:17 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:17.270421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf8bda-5de0-4852-a2c2-d2c57605541a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:17.388397 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acdf8bda-5de0-4852-a2c2-d2c57605541a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:17.422892 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29925e32-b4c1-4a2c-8d43-ca933e09d4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:17.423622 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29925e32-b4c1-4a2c-8d43-ca933e09d4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:17.423933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29925e32-b4c1-4a2c-8d43-ca933e09d4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:17.424214 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29925e32-b4c1-4a2c-8d43-ca933e09d4a8 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:17.424870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29925e32-b4c1-4a2c-8d43-ca933e09d4a8 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:17.426250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29925e32-b4c1-4a2c-8d43-ca933e09d4a8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:17.438319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-29925e32-b4c1-4a2c-8d43-ca933e09d4a8 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 15:55:17.438319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:17.438319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:55:17.438319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:55:17.438319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:55:17.438319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:55:17.438319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:55:17.438319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:17.438319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 15:55:17.438319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:17.438319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:17.438319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:17.438319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:17.438319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:17.438319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:17.438319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:17.438319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:17.439371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:17.439371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:17.439371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:17.439371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:17.439371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:17.439371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:17.439371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:55:17.439371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:55:17.439371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:55:17.439371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:55:17.439371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:17.439371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:17.439371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:17.439371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:17.439371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:17.439371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:17.440270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:17.440270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:17.440270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:17.440270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:17.440270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:17.440270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:17.440270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:17.440270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:17.440270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:17.440270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:17.440270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:55:17.440270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:17.440270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 15:55:17.440270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:17.444174 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 488/992] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:55:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:17.446033 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420484ae-33b8-4d08-b072-f5d0d7ea7f02 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:17.547961 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420484ae-33b8-4d08-b072-f5d0d7ea7f02 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:17.581537 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57a905-3d03-42e5-925a-9c1872f59642 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:17.582207 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57a905-3d03-42e5-925a-9c1872f59642 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:17.582494 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57a905-3d03-42e5-925a-9c1872f59642 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:17.582772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a57a905-3d03-42e5-925a-9c1872f59642 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:17.583485 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57a905-3d03-42e5-925a-9c1872f59642 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:17.583798 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57a905-3d03-42e5-925a-9c1872f59642 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:17.585265 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a57a905-3d03-42e5-925a-9c1872f59642 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:17.590410 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 505/993] 10.222.0.85 () {56 vars in 1112 bytes} [Wed May 25 15:55:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:17.595995 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29925e32-b4c1-4a2c-8d43-ca933e09d4a8 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:17.652296 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:17.696978 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29925e32-b4c1-4a2c-8d43-ca933e09d4a8 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:17.730601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c3ba6-1868-47b7-95c8-4699f9c47db1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:17.731275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c3ba6-1868-47b7-95c8-4699f9c47db1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:17.731607 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c3ba6-1868-47b7-95c8-4699f9c47db1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:17.731934 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-359c3ba6-1868-47b7-95c8-4699f9c47db1 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:17.732647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c3ba6-1868-47b7-95c8-4699f9c47db1 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:17.732960 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c3ba6-1868-47b7-95c8-4699f9c47db1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:17.734447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-359c3ba6-1868-47b7-95c8-4699f9c47db1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:17.739855 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 489/994] 10.222.0.85 () {56 vars in 1112 bytes} [Wed May 25 15:55:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:17.745739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57a905-3d03-42e5-925a-9c1872f59642 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:17.793982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:17.851074 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a57a905-3d03-42e5-925a-9c1872f59642 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:17.887191 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389c6652-360c-4515-ac2c-013c981ba597 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:17.887871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c6652-360c-4515-ac2c-013c981ba597 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:17.888161 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c6652-360c-4515-ac2c-013c981ba597 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:17.888435 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389c6652-360c-4515-ac2c-013c981ba597 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:17.889228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c6652-360c-4515-ac2c-013c981ba597 None admin] RBAC: Adding query filter params (interface=public, service_id=e3b23876f74f4262bd70af12b6198218, region_id=RegionOne) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:17.889543 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c6652-360c-4515-ac2c-013c981ba597 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:17.890921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389c6652-360c-4515-ac2c-013c981ba597 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:17.896394 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 506/995] 10.222.0.85 () {56 vars in 1251 bytes} [Wed May 25 15:55:17 2022] GET /identity/v3/endpoints?service_id=e3b23876f74f4262bd70af12b6198218&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:17.901815 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c3ba6-1868-47b7-95c8-4699f9c47db1 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:18.002048 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-359c3ba6-1868-47b7-95c8-4699f9c47db1 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:18.021820 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:18.038459 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2a3b66-3d31-40a5-8578-55684d6a07ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:18.039136 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a3b66-3d31-40a5-8578-55684d6a07ac None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:18.039429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a3b66-3d31-40a5-8578-55684d6a07ac None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:18.039715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2a3b66-3d31-40a5-8578-55684d6a07ac None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:18.040425 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a3b66-3d31-40a5-8578-55684d6a07ac None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:18.041848 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2a3b66-3d31-40a5-8578-55684d6a07ac None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:18.055847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 490/996] 10.222.0.85 () {56 vars in 1087 bytes} [Wed May 25 15:55:17 2022] GET /identity/v3/services => generated 1727 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:55:18.314065 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246cf43-c197-4c6f-97d8-b0ed1cfa52af None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:18.314524 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246cf43-c197-4c6f-97d8-b0ed1cfa52af None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:18.314630 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b246cf43-c197-4c6f-97d8-b0ed1cfa52af None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:18.316185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 507/997] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:18.321349 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c86b06-1cf1-4e95-a26b-9e09ad276cbf None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:18.321767 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c86b06-1cf1-4e95-a26b-9e09ad276cbf None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:18.321958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c86b06-1cf1-4e95-a26b-9e09ad276cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:18.339302 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-68c86b06-1cf1-4e95-a26b-9e09ad276cbf None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:18.340534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-68c86b06-1cf1-4e95-a26b-9e09ad276cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:18.411041 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 491/998] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:18.434179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246cf43-c197-4c6f-97d8-b0ed1cfa52af None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:18.440143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb1803-d228-45e4-86c5-a7ac12039f7e None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:18.440580 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb1803-d228-45e4-86c5-a7ac12039f7e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:18.440893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6bb1803-d228-45e4-86c5-a7ac12039f7e None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:18.442430 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 492/999] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:18.447972 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372acb47-139b-499f-ae38-081bca04db59 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:18.448270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372acb47-139b-499f-ae38-081bca04db59 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:18.448602 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-372acb47-139b-499f-ae38-081bca04db59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:18.467199 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-372acb47-139b-499f-ae38-081bca04db59 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:18.468425 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-372acb47-139b-499f-ae38-081bca04db59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:18.543981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 493/1000] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:18 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:55:18.549569 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b246cf43-c197-4c6f-97d8-b0ed1cfa52af None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:18.560719 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372acb47-139b-499f-ae38-081bca04db59 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:18.561236 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:18.588487 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e535200-d654-4d48-9359-988c60c21ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:18.589326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e535200-d654-4d48-9359-988c60c21ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:18.589617 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e535200-d654-4d48-9359-988c60c21ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:18.589903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e535200-d654-4d48-9359-988c60c21ac6 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:18.590780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e535200-d654-4d48-9359-988c60c21ac6 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:18.591992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e535200-d654-4d48-9359-988c60c21ac6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:18.604508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8e535200-d654-4d48-9359-988c60c21ac6 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 15:55:18.604508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:18.604508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:55:18.604508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:55:18.604508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:55:18.604508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:55:18.604508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:55:18.604508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:18.604508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 15:55:18.604508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:18.604508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:18.604508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:18.604508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:18.604508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:18.604508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:18.604508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:18.604508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:18.605564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:18.605564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:18.605564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:18.605564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:18.605564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:18.605564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:18.605564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:55:18.605564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:55:18.605564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:55:18.605564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:55:18.605564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:18.605564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:18.605564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:18.605564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:18.605564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:18.605564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:18.606434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:18.606434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:18.606434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:18.606434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:18.606434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:18.606434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:18.606434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:18.606434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:18.606434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:18.606434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:18.606434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:55:18.606434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:18.606434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. May 25 15:55:18.606434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:18.606434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 508/1001] 10.222.0.85 () {56 vars in 1126 bytes} [Wed May 25 15:55:18 2022] GET /identity/v3/services/object-store => generated 93 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:18.611646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e535200-d654-4d48-9359-988c60c21ac6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:18.612210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:18.668548 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-372acb47-139b-499f-ae38-081bca04db59 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:18.702203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662f8ce-78fe-4020-8ccb-209a90c79767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:18.703280 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662f8ce-78fe-4020-8ccb-209a90c79767 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:18.703575 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662f8ce-78fe-4020-8ccb-209a90c79767 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:18.703883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5662f8ce-78fe-4020-8ccb-209a90c79767 None admin] PATH_INFO: `/v3/services/image` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:18.704545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662f8ce-78fe-4020-8ccb-209a90c79767 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:18.706307 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5662f8ce-78fe-4020-8ccb-209a90c79767 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:18.718059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-5662f8ce-78fe-4020-8ccb-209a90c79767 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 15:55:18.718059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:18.718059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:55:18.718059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:55:18.718059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:55:18.718059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:55:18.718059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:55:18.718059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:18.718059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 15:55:18.718059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:18.718059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:18.718059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:18.718059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:18.718059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:18.718059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:18.718059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:18.718059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:18.719091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:18.719091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:18.719091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:18.719091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:18.719091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:18.719091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:18.719091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:55:18.719091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:55:18.719091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:55:18.719091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:55:18.719091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:18.719091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:18.719091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:18.719091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:18.719091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:18.719091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:18.719979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:18.719979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:18.719979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:18.719979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:18.719979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:18.719979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:18.719979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:18.719979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:18.719979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:18.719979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:18.719979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:55:18.719979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:18.719979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 15:55:18.719979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:18.720757 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e535200-d654-4d48-9359-988c60c21ac6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:18.720834 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 494/1002] 10.222.0.85 () {56 vars in 1105 bytes} [Wed May 25 15:55:18 2022] GET /identity/v3/services/image => generated 86 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:18.725415 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662f8ce-78fe-4020-8ccb-209a90c79767 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:18.759985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d80026-6e6b-4fe6-8985-c67f75455002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:18.760674 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d80026-6e6b-4fe6-8985-c67f75455002 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:18.760964 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d80026-6e6b-4fe6-8985-c67f75455002 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:18.761245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36d80026-6e6b-4fe6-8985-c67f75455002 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:18.761983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d80026-6e6b-4fe6-8985-c67f75455002 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:18.762299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d80026-6e6b-4fe6-8985-c67f75455002 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:18.763701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36d80026-6e6b-4fe6-8985-c67f75455002 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:18.773632 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 509/1003] 10.222.0.85 () {56 vars in 1122 bytes} [Wed May 25 15:55:18 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:18.774742 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d80026-6e6b-4fe6-8985-c67f75455002 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:18.819770 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:18.844159 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5662f8ce-78fe-4020-8ccb-209a90c79767 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:18.854737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:18.881777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905849da-c670-44ac-b5ec-c28ed259bca9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:18.882461 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905849da-c670-44ac-b5ec-c28ed259bca9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:18.882750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905849da-c670-44ac-b5ec-c28ed259bca9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:18.883035 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905849da-c670-44ac-b5ec-c28ed259bca9 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:18.883747 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905849da-c670-44ac-b5ec-c28ed259bca9 None admin] RBAC: Adding query filter params (name=image) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:18.884098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905849da-c670-44ac-b5ec-c28ed259bca9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:18.884515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36d80026-6e6b-4fe6-8985-c67f75455002 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:18.885537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905849da-c670-44ac-b5ec-c28ed259bca9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:18.890575 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 495/1004] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:18 2022] GET /identity/v3/services?name=image => generated 122 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:18.896062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905849da-c670-44ac-b5ec-c28ed259bca9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:18.923201 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75504d2b-63f9-4d27-a785-5f5282d64a3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:18.923955 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75504d2b-63f9-4d27-a785-5f5282d64a3f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:18.924249 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75504d2b-63f9-4d27-a785-5f5282d64a3f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:18.924530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75504d2b-63f9-4d27-a785-5f5282d64a3f None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:18.925239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75504d2b-63f9-4d27-a785-5f5282d64a3f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:18.925557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75504d2b-63f9-4d27-a785-5f5282d64a3f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:18.926978 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75504d2b-63f9-4d27-a785-5f5282d64a3f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:18.932034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 510/1005] 10.222.0.85 () {56 vars in 1122 bytes} [Wed May 25 15:55:18 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:18.937584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75504d2b-63f9-4d27-a785-5f5282d64a3f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:19.004429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905849da-c670-44ac-b5ec-c28ed259bca9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:19.038328 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f0bf7f-5f10-40da-b2ee-7e0ed72f51e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:19.039078 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0bf7f-5f10-40da-b2ee-7e0ed72f51e3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:19.039432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0bf7f-5f10-40da-b2ee-7e0ed72f51e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:19.039783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f0bf7f-5f10-40da-b2ee-7e0ed72f51e3 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:19.040579 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f0bf7f-5f10-40da-b2ee-7e0ed72f51e3 None admin] RBAC: Adding query filter params (type=image) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:19.040960 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f0bf7f-5f10-40da-b2ee-7e0ed72f51e3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:19.042016 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75504d2b-63f9-4d27-a785-5f5282d64a3f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:19.042498 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f0bf7f-5f10-40da-b2ee-7e0ed72f51e3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:19.043690 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:19.055850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 496/1006] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:18 2022] GET /identity/v3/services?type=image => generated 122 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:19.081709 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c24f64-4fee-411c-bc11-2175f02a4201 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:19.082432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c24f64-4fee-411c-bc11-2175f02a4201 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:19.082794 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c24f64-4fee-411c-bc11-2175f02a4201 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:19.083148 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c24f64-4fee-411c-bc11-2175f02a4201 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:19.084022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c24f64-4fee-411c-bc11-2175f02a4201 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:19.085525 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c24f64-4fee-411c-bc11-2175f02a4201 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:19.105344 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 511/1007] 10.222.0.85 () {60 vars in 1144 bytes} [Wed May 25 15:55:18 2022] POST /identity/v3/endpoints => generated 325 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:55:20.260873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ba399-de67-4f25-a2da-a1b30cc7f0aa None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:20.261412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ba399-de67-4f25-a2da-a1b30cc7f0aa None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:20.261412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51ba399-de67-4f25-a2da-a1b30cc7f0aa None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:20.262827 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 497/1008] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:20.268252 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474dee9-8e2b-49d3-b65c-92e5fe8d7779 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:20.268769 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474dee9-8e2b-49d3-b65c-92e5fe8d7779 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:20.268769 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474dee9-8e2b-49d3-b65c-92e5fe8d7779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:20.286329 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8474dee9-8e2b-49d3-b65c-92e5fe8d7779 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:20.287525 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8474dee9-8e2b-49d3-b65c-92e5fe8d7779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:20.366105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 512/1009] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:20 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:20.388729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ba399-de67-4f25-a2da-a1b30cc7f0aa None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:20.428072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:20.495667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51ba399-de67-4f25-a2da-a1b30cc7f0aa None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:20.528958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e8ad2-9cc0-4a7a-977b-355a8ae9dd66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:20.529698 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e8ad2-9cc0-4a7a-977b-355a8ae9dd66 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:20.529979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e8ad2-9cc0-4a7a-977b-355a8ae9dd66 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:20.530253 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1e8ad2-9cc0-4a7a-977b-355a8ae9dd66 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:20.530905 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e8ad2-9cc0-4a7a-977b-355a8ae9dd66 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:20.532420 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1e8ad2-9cc0-4a7a-977b-355a8ae9dd66 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:20.544527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-9e1e8ad2-9cc0-4a7a-977b-355a8ae9dd66 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. May 25 15:55:20.544527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:20.544527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:55:20.544527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:55:20.544527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:55:20.544527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:55:20.544527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:55:20.544527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:20.544527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 15:55:20.544527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:20.544527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:20.544527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:20.544527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:20.544527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:20.544527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:20.544527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:20.544527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:20.545539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:20.545539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:20.545539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:20.545539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:20.545539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:20.545539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:20.545539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:55:20.545539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:55:20.545539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:55:20.545539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:55:20.545539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:20.545539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:20.545539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:20.545539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:20.545539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:20.545539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:20.552019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:20.552019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:20.552019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:20.552019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:20.552019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:20.552019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:20.552019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:20.552019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:20.552019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:20.552019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:20.552019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:55:20.552019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:20.552019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. May 25 15:55:20.552019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:20.552019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 498/1010] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:55:20 2022] GET /identity/v3/services/compute => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:20.552883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474dee9-8e2b-49d3-b65c-92e5fe8d7779 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:20.642225 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:20.657716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474dee9-8e2b-49d3-b65c-92e5fe8d7779 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:20.691789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16607c-8743-4251-903d-6adde3943249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:20.692530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16607c-8743-4251-903d-6adde3943249 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:20.692815 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16607c-8743-4251-903d-6adde3943249 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:20.693159 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e16607c-8743-4251-903d-6adde3943249 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:20.693888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16607c-8743-4251-903d-6adde3943249 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:20.694197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16607c-8743-4251-903d-6adde3943249 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:20.695579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e16607c-8743-4251-903d-6adde3943249 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:20.700857 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 513/1011] 10.222.0.85 () {56 vars in 1112 bytes} [Wed May 25 15:55:20 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:20.706776 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e8ad2-9cc0-4a7a-977b-355a8ae9dd66 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:20.808243 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:20.812423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1e8ad2-9cc0-4a7a-977b-355a8ae9dd66 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:20.848051 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7bd80d1-8b81-4665-a4f4-f4b050446c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:20.848723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd80d1-8b81-4665-a4f4-f4b050446c82 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:20.849015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd80d1-8b81-4665-a4f4-f4b050446c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:20.849297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7bd80d1-8b81-4665-a4f4-f4b050446c82 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:20.850012 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd80d1-8b81-4665-a4f4-f4b050446c82 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:20.850323 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd80d1-8b81-4665-a4f4-f4b050446c82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:20.851724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7bd80d1-8b81-4665-a4f4-f4b050446c82 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:20.856943 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 499/1012] 10.222.0.85 () {56 vars in 1112 bytes} [Wed May 25 15:55:20 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:20.862502 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16607c-8743-4251-903d-6adde3943249 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:20.969689 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e16607c-8743-4251-903d-6adde3943249 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:21.005982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f87da-d37a-4552-a112-bec1450a2c3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:21.006648 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f87da-d37a-4552-a112-bec1450a2c3c None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:21.006931 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f87da-d37a-4552-a112-bec1450a2c3c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:21.007208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f87da-d37a-4552-a112-bec1450a2c3c None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:21.008039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f87da-d37a-4552-a112-bec1450a2c3c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:21.009432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8f87da-d37a-4552-a112-bec1450a2c3c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:21.029323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 514/1013] 10.222.0.85 () {60 vars in 1144 bytes} [Wed May 25 15:55:20 2022] POST /identity/v3/endpoints => generated 334 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:55:21.261083 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186d6cb-b1b4-4b95-b660-b60a3757ae61 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:21.261654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186d6cb-b1b4-4b95-b660-b60a3757ae61 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:21.261654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b186d6cb-b1b4-4b95-b660-b60a3757ae61 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:21.263183 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 500/1014] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:21.268456 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1fb6af-e337-45c4-8284-f70edf35c66b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:21.268895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1fb6af-e337-45c4-8284-f70edf35c66b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:21.268987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1fb6af-e337-45c4-8284-f70edf35c66b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:21.286397 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2b1fb6af-e337-45c4-8284-f70edf35c66b None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:21.287576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2b1fb6af-e337-45c4-8284-f70edf35c66b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:21.289655 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:21.335218 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b69f0-362e-40f0-8a12-dde034a1b364 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:21.335513 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b69f0-362e-40f0-8a12-dde034a1b364 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:21.335800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430b69f0-362e-40f0-8a12-dde034a1b364 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:21.337236 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 501/1015] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:21.342731 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582dcf65-3a91-431f-9fff-1f641ac04347 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:21.343024 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582dcf65-3a91-431f-9fff-1f641ac04347 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:21.343306 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-582dcf65-3a91-431f-9fff-1f641ac04347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:21.363315 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-582dcf65-3a91-431f-9fff-1f641ac04347 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:21.364661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-582dcf65-3a91-431f-9fff-1f641ac04347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:21.376269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 515/1016] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:21 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:21.398356 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1fb6af-e337-45c4-8284-f70edf35c66b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:21.433312 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 502/1017] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:21 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:55:21.458930 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582dcf65-3a91-431f-9fff-1f641ac04347 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:21.516872 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1fb6af-e337-45c4-8284-f70edf35c66b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:21.554493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f77c1-e2ef-4e90-94ac-5ba5cc0a3378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:21.555234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f77c1-e2ef-4e90-94ac-5ba5cc0a3378 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:21.555570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f77c1-e2ef-4e90-94ac-5ba5cc0a3378 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:21.555939 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e1f77c1-e2ef-4e90-94ac-5ba5cc0a3378 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:21.556680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f77c1-e2ef-4e90-94ac-5ba5cc0a3378 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:21.557364 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:21.558162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e1f77c1-e2ef-4e90-94ac-5ba5cc0a3378 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:21.560356 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:21.565496 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-582dcf65-3a91-431f-9fff-1f641ac04347 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:21.577520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 516/1018] 10.222.0.85 () {60 vars in 1141 bytes} [Wed May 25 15:55:21 2022] POST /identity/v3/services => generated 242 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:55:21.612806 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7b4ae2-275c-4654-bef8-0cd51eed36be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:21.613686 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b4ae2-275c-4654-bef8-0cd51eed36be None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:21.614019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b4ae2-275c-4654-bef8-0cd51eed36be None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:21.614309 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7b4ae2-275c-4654-bef8-0cd51eed36be None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:21.615687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b4ae2-275c-4654-bef8-0cd51eed36be None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:21.617266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7b4ae2-275c-4654-bef8-0cd51eed36be None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:21.619273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 503/1019] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:21.625068 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f77c1-e2ef-4e90-94ac-5ba5cc0a3378 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:21.742430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e1f77c1-e2ef-4e90-94ac-5ba5cc0a3378 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:21.782102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf1f5052-10be-4ce3-a4ec-7e8d482b274f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:21.782710 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f5052-10be-4ce3-a4ec-7e8d482b274f None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:21.782995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f5052-10be-4ce3-a4ec-7e8d482b274f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:21.783275 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1f5052-10be-4ce3-a4ec-7e8d482b274f None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:21.784034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f5052-10be-4ce3-a4ec-7e8d482b274f None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:21.785533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf1f5052-10be-4ce3-a4ec-7e8d482b274f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:21.812835 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 517/1020] 10.222.0.85 () {60 vars in 1141 bytes} [Wed May 25 15:55:21 2022] POST /identity/v3/projects => generated 311 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:55:23.361100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddeeeb-276b-48f7-a61e-dfaef59e87d8 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:23.361680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddeeeb-276b-48f7-a61e-dfaef59e87d8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:23.361680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ddeeeb-276b-48f7-a61e-dfaef59e87d8 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:23.363022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 504/1021] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:23.368696 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f5ee6-5586-490f-9b45-f4aca708b1ee None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:23.369168 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f5ee6-5586-490f-9b45-f4aca708b1ee None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:23.369301 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952f5ee6-5586-490f-9b45-f4aca708b1ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:23.387070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-952f5ee6-5586-490f-9b45-f4aca708b1ee None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:23.388307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-952f5ee6-5586-490f-9b45-f4aca708b1ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:23.435755 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:23.461950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 518/1022] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:23.485365 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddeeeb-276b-48f7-a61e-dfaef59e87d8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:23.592397 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ddeeeb-276b-48f7-a61e-dfaef59e87d8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:23.629898 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35acdf21-615c-4545-ab92-18d22d4edf6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:23.630735 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acdf21-615c-4545-ab92-18d22d4edf6a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:23.631022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acdf21-615c-4545-ab92-18d22d4edf6a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:23.631303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35acdf21-615c-4545-ab92-18d22d4edf6a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:23.641903 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35acdf21-615c-4545-ab92-18d22d4edf6a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:55:23.650230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-35acdf21-615c-4545-ab92-18d22d4edf6a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 15:55:23.650230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:23.650230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:23.650230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:23.650230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:23.650230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:23.650230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:23.650230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:23.650230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:23.650230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:23.650230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:23.650230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:23.650230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:55:23.650230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:55:23.650230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:55:23.650230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:55:23.651184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:55:23.651184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:55:23.651184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:55:23.651184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:55:23.651184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:23.651184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:23.651184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:23.651184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:23.651184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:23.651184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:23.651184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:23.651184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:23.651184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:23.651184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:23.651184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:23.651184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:23.652439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:23.652439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:23.652439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:23.652439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:23.652439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:55:23.652439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:55:23.652439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:55:23.652439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:55:23.652439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:55:23.652439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:55:23.652439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 15:55:23.652439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:23.656171 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 505/1023] 10.222.0.85 () {56 vars in 1120 bytes} [Wed May 25 15:55:23 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:23.657587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f5ee6-5586-490f-9b45-f4aca708b1ee None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:23.761826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952f5ee6-5586-490f-9b45-f4aca708b1ee None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:23.777895 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc292b-49d1-4458-a0a5-baee7f8992e7 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:23.778185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc292b-49d1-4458-a0a5-baee7f8992e7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:23.778492 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bc292b-49d1-4458-a0a5-baee7f8992e7 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:23.780037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 506/1024] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) May 25 15:55:23.784739 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50073896-fb48-4959-8736-9b49d0496348 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:23.785006 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50073896-fb48-4959-8736-9b49d0496348 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:23.785314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50073896-fb48-4959-8736-9b49d0496348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:23.799031 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3e1ed-c449-4d1c-9c24-21f55d586c95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:23.799718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3e1ed-c449-4d1c-9c24-21f55d586c95 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:23.800039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3e1ed-c449-4d1c-9c24-21f55d586c95 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:23.800325 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c3e1ed-c449-4d1c-9c24-21f55d586c95 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:23.801088 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3e1ed-c449-4d1c-9c24-21f55d586c95 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:23.801407 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3e1ed-c449-4d1c-9c24-21f55d586c95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:23.802114 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-50073896-fb48-4959-8736-9b49d0496348 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:23.802796 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c3e1ed-c449-4d1c-9c24-21f55d586c95 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:23.803289 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-50073896-fb48-4959-8736-9b49d0496348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:23.811297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 519/1025] 10.222.0.85 () {56 vars in 1115 bytes} [Wed May 25 15:55:23 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:23.817092 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:23.817219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3e1ed-c449-4d1c-9c24-21f55d586c95 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:23.862001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:23.879813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 507/1026] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:23 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:55:23.901741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50073896-fb48-4959-8736-9b49d0496348 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:23.936208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c3e1ed-c449-4d1c-9c24-21f55d586c95 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:23.971702 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e941be-e0b0-4c6a-83b6-faa34f1a83c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:23.972553 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e941be-e0b0-4c6a-83b6-faa34f1a83c2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:23.972829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e941be-e0b0-4c6a-83b6-faa34f1a83c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:23.973111 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e941be-e0b0-4c6a-83b6-faa34f1a83c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:23.984149 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e941be-e0b0-4c6a-83b6-faa34f1a83c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:23.985615 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e941be-e0b0-4c6a-83b6-faa34f1a83c2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:24.003528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-a6e941be-e0b0-4c6a-83b6-faa34f1a83c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:24.003528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:24.003528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:24.003528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:24.003528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:24.003528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:24.003528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:24.003528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:24.003528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:24.003528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:24.003528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:24.003528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:24.003528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:55:24.003528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:55:24.003528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:55:24.004666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:55:24.004666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:24.004666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:24.004666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:24.004666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:24.004666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:24.004666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:24.004666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:24.004666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:24.004666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:24.004666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:24.004666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:24.004666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:24.004666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:24.004666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:24.004666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:24.004666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:24.005601 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:55:24.005601 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:55:24.005601 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:24.005601 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:24.005601 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 520/1027] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:55:24.010548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e941be-e0b0-4c6a-83b6-faa34f1a83c2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:24.017627 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50073896-fb48-4959-8736-9b49d0496348 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:24.057498 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b47368e-4000-4a5a-ab5e-6b20bb824ee8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:24.058353 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47368e-4000-4a5a-ab5e-6b20bb824ee8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:24.058642 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47368e-4000-4a5a-ab5e-6b20bb824ee8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:24.058939 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b47368e-4000-4a5a-ab5e-6b20bb824ee8 None admin] PATH_INFO: `/v3/services/image` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:24.059581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47368e-4000-4a5a-ab5e-6b20bb824ee8 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:24.061014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b47368e-4000-4a5a-ab5e-6b20bb824ee8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:24.072569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-7b47368e-4000-4a5a-ab5e-6b20bb824ee8 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 15:55:24.072569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:24.072569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:55:24.072569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:55:24.072569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:55:24.072569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:55:24.072569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:55:24.072569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:24.072569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 15:55:24.072569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:24.072569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:24.072569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:24.072569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:24.072569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:24.072569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:24.072569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:24.072569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:24.073761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:24.073761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:24.073761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:24.073761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:24.073761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:24.073761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:24.073761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:55:24.073761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:55:24.073761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:55:24.073761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:55:24.073761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:24.073761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:24.073761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:24.073761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:24.073761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:24.073761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:24.074740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:24.074740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:24.074740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:24.074740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:24.074740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:24.074740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:24.074740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:24.074740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:24.074740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:24.074740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:24.074740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:55:24.074740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:24.074740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 15:55:24.074740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:24.074740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 508/1028] 10.222.0.85 () {56 vars in 1105 bytes} [Wed May 25 15:55:23 2022] GET /identity/v3/services/image => generated 86 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:24.075716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146d87a-2e62-46e2-8077-bf41c53c5fa7 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:24.075716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146d87a-2e62-46e2-8077-bf41c53c5fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:24.075716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f146d87a-2e62-46e2-8077-bf41c53c5fa7 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:24.077202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 509/1029] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:24.080091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146d87a-2e62-46e2-8077-bf41c53c5fa7 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:24.095147 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:24.114492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:24.124438 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e941be-e0b0-4c6a-83b6-faa34f1a83c2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:24.157492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab3a1e68-bd74-4a3c-8f70-8777857ea144 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:24.158156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a1e68-bd74-4a3c-8f70-8777857ea144 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:24.158441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a1e68-bd74-4a3c-8f70-8777857ea144 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:24.158721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab3a1e68-bd74-4a3c-8f70-8777857ea144 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:24.159441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a1e68-bd74-4a3c-8f70-8777857ea144 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:24.159752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a1e68-bd74-4a3c-8f70-8777857ea144 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:24.161156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab3a1e68-bd74-4a3c-8f70-8777857ea144 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:24.177486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 521/1030] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:55:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:24.177486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f484c-e83f-4422-85b9-602f57e3ba54 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:24.177486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f484c-e83f-4422-85b9-602f57e3ba54 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:24.177486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8f484c-e83f-4422-85b9-602f57e3ba54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:24.184882 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f146d87a-2e62-46e2-8077-bf41c53c5fa7 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:24.191587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dc8f484c-e83f-4422-85b9-602f57e3ba54 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:24.192828 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dc8f484c-e83f-4422-85b9-602f57e3ba54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:24.218838 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c8c4f-40ae-426f-8094-65a3361fb552 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:24.219513 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c8c4f-40ae-426f-8094-65a3361fb552 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:24.219912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c8c4f-40ae-426f-8094-65a3361fb552 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:24.220193 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-162c8c4f-40ae-426f-8094-65a3361fb552 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:24.220929 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c8c4f-40ae-426f-8094-65a3361fb552 None admin] RBAC: Adding query filter params (name=image) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:24.221240 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c8c4f-40ae-426f-8094-65a3361fb552 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:24.222631 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-162c8c4f-40ae-426f-8094-65a3361fb552 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:24.227491 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 510/1031] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:24 2022] GET /identity/v3/services?name=image => generated 122 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:24.228525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c8c4f-40ae-426f-8094-65a3361fb552 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:24.264533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 522/1032] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:24 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:24.265377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f484c-e83f-4422-85b9-602f57e3ba54 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:24.311231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:24.334640 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-162c8c4f-40ae-426f-8094-65a3361fb552 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:24.347814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:24.367348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8f484c-e83f-4422-85b9-602f57e3ba54 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:24.369337 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a8434-6686-47a4-99a6-5827c475f54b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:24.370153 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a8434-6686-47a4-99a6-5827c475f54b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:24.370441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a8434-6686-47a4-99a6-5827c475f54b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:24.370726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632a8434-6686-47a4-99a6-5827c475f54b None admin] PATH_INFO: `/v3/users/nova` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:24.377847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a8434-6686-47a4-99a6-5827c475f54b None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:24.379303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632a8434-6686-47a4-99a6-5827c475f54b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:24.394065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-632a8434-6686-47a4-99a6-5827c475f54b None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 15:55:24.394065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:24.394065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:24.394065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:24.394065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:24.394065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:24.394065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:24.394065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:24.394065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:24.394065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:24.394065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:24.394065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:24.394065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:55:24.394065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:55:24.394065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:55:24.394065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:24.395135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:24.396669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:24.396669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:24.396669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:55:24.396669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:55:24.396669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:55:24.396669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:55:24.396669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:55:24.396669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:55:24.396669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 15:55:24.396669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:24.396669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 511/1033] 10.222.0.85 () {56 vars in 1093 bytes} [Wed May 25 15:55:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:24.396669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a8434-6686-47a4-99a6-5827c475f54b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:24.403570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c995773-c434-4558-9f68-daa1e37e01c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:24.404288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c995773-c434-4558-9f68-daa1e37e01c6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:24.404579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c995773-c434-4558-9f68-daa1e37e01c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:24.404863 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c995773-c434-4558-9f68-daa1e37e01c6 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:24.405589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c995773-c434-4558-9f68-daa1e37e01c6 None admin] RBAC: Adding query filter params (type=image) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:24.405896 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c995773-c434-4558-9f68-daa1e37e01c6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:24.407289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c995773-c434-4558-9f68-daa1e37e01c6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:24.417327 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 523/1034] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:24 2022] GET /identity/v3/services?type=image => generated 350 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:24.417327 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c995773-c434-4558-9f68-daa1e37e01c6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:24.498554 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632a8434-6686-47a4-99a6-5827c475f54b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:24.515574 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c995773-c434-4558-9f68-daa1e37e01c6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:24.533258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8bdca-ca90-48cf-b091-2d82b847cd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:24.534044 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8bdca-ca90-48cf-b091-2d82b847cd65 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:24.534353 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8bdca-ca90-48cf-b091-2d82b847cd65 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:24.534633 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec8bdca-ca90-48cf-b091-2d82b847cd65 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:24.535801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8bdca-ca90-48cf-b091-2d82b847cd65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:24.537295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec8bdca-ca90-48cf-b091-2d82b847cd65 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:24.538986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 512/1035] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:24 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:24.539753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8bdca-ca90-48cf-b091-2d82b847cd65 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:24.543726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:24.556454 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e93dec-c67b-497a-b0dd-766cb6ebe2c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:24.557141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e93dec-c67b-497a-b0dd-766cb6ebe2c4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:24.557434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e93dec-c67b-497a-b0dd-766cb6ebe2c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:24.557721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97e93dec-c67b-497a-b0dd-766cb6ebe2c4 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:24.558570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e93dec-c67b-497a-b0dd-766cb6ebe2c4 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:24.558891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e93dec-c67b-497a-b0dd-766cb6ebe2c4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:24.560396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97e93dec-c67b-497a-b0dd-766cb6ebe2c4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:24.579000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 524/1036] 10.222.0.85 () {56 vars in 1133 bytes} [Wed May 25 15:55:24 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:24.580711 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e93dec-c67b-497a-b0dd-766cb6ebe2c4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:24.583345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:24.640453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec8bdca-ca90-48cf-b091-2d82b847cd65 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:24.673835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b43aa4-f526-47f9-af47-c3b1dc9a6a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:24.674487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b43aa4-f526-47f9-af47-c3b1dc9a6a66 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:24.674771 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b43aa4-f526-47f9-af47-c3b1dc9a6a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:24.675049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b43aa4-f526-47f9-af47-c3b1dc9a6a66 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:24.675866 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b43aa4-f526-47f9-af47-c3b1dc9a6a66 None admin] RBAC: Adding query filter params (interface=public, service_id=e865fdc887384a1e88813fd05bc71b01, region_id=RegionOne) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:24.676194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b43aa4-f526-47f9-af47-c3b1dc9a6a66 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:24.677569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b43aa4-f526-47f9-af47-c3b1dc9a6a66 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:24.677985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97e93dec-c67b-497a-b0dd-766cb6ebe2c4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:24.682608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 513/1037] 10.222.0.85 () {56 vars in 1251 bytes} [Wed May 25 15:55:24 2022] GET /identity/v3/endpoints?service_id=e865fdc887384a1e88813fd05bc71b01&interface=public®ion_id=RegionOne => generated 194 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:24.684611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b43aa4-f526-47f9-af47-c3b1dc9a6a66 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:24.711801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabbe68-a7e0-4fb6-b333-227af48c4f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:24.712505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabbe68-a7e0-4fb6-b333-227af48c4f8f None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:24.712818 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabbe68-a7e0-4fb6-b333-227af48c4f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:24.713098 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbabbe68-a7e0-4fb6-b333-227af48c4f8f None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:24.713811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbabbe68-a7e0-4fb6-b333-227af48c4f8f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:24.715248 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbabbe68-a7e0-4fb6-b333-227af48c4f8f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:24.737345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 525/1038] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:55:24 2022] POST /identity/v3/users => generated 285 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:55:24.738160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabbe68-a7e0-4fb6-b333-227af48c4f8f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:24.751062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:24.791841 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b43aa4-f526-47f9-af47-c3b1dc9a6a66 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:24.816496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:24.831725 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd03b33-1aa0-4892-b99f-655bb83df7f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:24.832446 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd03b33-1aa0-4892-b99f-655bb83df7f6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:24.832738 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd03b33-1aa0-4892-b99f-655bb83df7f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:24.833020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd03b33-1aa0-4892-b99f-655bb83df7f6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:24.865554 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbabbe68-a7e0-4fb6-b333-227af48c4f8f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:24.904087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5a282-5a2f-437d-84dc-64b0a5ea8064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:24.905405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5a282-5a2f-437d-84dc-64b0a5ea8064 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:24.905774 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5a282-5a2f-437d-84dc-64b0a5ea8064 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:24.906060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade5a282-5a2f-437d-84dc-64b0a5ea8064 None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:24.906778 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5a282-5a2f-437d-84dc-64b0a5ea8064 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:24.908241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ade5a282-5a2f-437d-84dc-64b0a5ea8064 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:24.908696 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd03b33-1aa0-4892-b99f-655bb83df7f6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:24.910507 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd03b33-1aa0-4892-b99f-655bb83df7f6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:24.916569 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 526/1039] 10.222.0.85 () {56 vars in 1087 bytes} [Wed May 25 15:55:24 2022] GET /identity/v3/services => generated 1957 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:55:24.989926 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:25.029836 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 514/1040] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:55:24 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:55:25.036116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5a282-5a2f-437d-84dc-64b0a5ea8064 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:25.146398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade5a282-5a2f-437d-84dc-64b0a5ea8064 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:25.169745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:25.182464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06631c-b350-48b3-b9eb-b6fce51a9afa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:25.183280 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06631c-b350-48b3-b9eb-b6fce51a9afa None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:25.183568 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06631c-b350-48b3-b9eb-b6fce51a9afa None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:25.183856 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df06631c-b350-48b3-b9eb-b6fce51a9afa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:25.195060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06631c-b350-48b3-b9eb-b6fce51a9afa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:25.196569 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df06631c-b350-48b3-b9eb-b6fce51a9afa None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:25.214587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-df06631c-b350-48b3-b9eb-b6fce51a9afa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:25.214587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:25.214587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:25.214587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:25.214587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:25.214587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:25.214587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:25.214587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:25.214587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:25.214587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:25.214587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:25.214587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:25.214587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:55:25.214587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:55:25.214587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:55:25.215523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:55:25.215523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:25.215523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:25.215523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:25.215523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:25.215523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:25.215523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:25.215523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:25.215523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:25.215523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:25.215523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:25.215523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:25.215523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:25.215523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:25.215523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:25.215523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:25.215523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:25.216463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:55:25.216463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:55:25.216463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:25.216463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:25.216463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 527/1041] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:25.225108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd03b33-1aa0-4892-b99f-655bb83df7f6 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:25.336484 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd03b33-1aa0-4892-b99f-655bb83df7f6 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:25.375637 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb2717-e2bc-40f1-ab7f-a4d326fda4e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:25.376368 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb2717-e2bc-40f1-ab7f-a4d326fda4e2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:25.376638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb2717-e2bc-40f1-ab7f-a4d326fda4e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:25.376942 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbfb2717-e2bc-40f1-ab7f-a4d326fda4e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:25.377672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb2717-e2bc-40f1-ab7f-a4d326fda4e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:25.377987 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb2717-e2bc-40f1-ab7f-a4d326fda4e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:25.379370 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbfb2717-e2bc-40f1-ab7f-a4d326fda4e2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:25.396163 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 515/1042] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:55:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:25.397246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06631c-b350-48b3-b9eb-b6fce51a9afa None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:25.499916 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df06631c-b350-48b3-b9eb-b6fce51a9afa None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:25.533468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3abac-564d-4a77-a12e-9892d1ccef6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:25.534277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3abac-564d-4a77-a12e-9892d1ccef6a None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:25.534580 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3abac-564d-4a77-a12e-9892d1ccef6a None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:25.534862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c3abac-564d-4a77-a12e-9892d1ccef6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:25.541654 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3abac-564d-4a77-a12e-9892d1ccef6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:25.543122 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60c3abac-564d-4a77-a12e-9892d1ccef6a None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:25.557405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-60c3abac-564d-4a77-a12e-9892d1ccef6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:25.557405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:25.557405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:25.557405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:25.557405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:25.557405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:25.557405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:25.557405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:25.557405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:25.557405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:25.557405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:25.557405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:25.557405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:55:25.557405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:55:25.557405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:55:25.558336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:55:25.558336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:25.558336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:25.558336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:25.558336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:25.558336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:25.558336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:25.558336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:25.558336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:25.558336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:25.558336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:25.558336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:25.558336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:25.558336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:25.558336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:25.558336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:25.558336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:25.564208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:55:25.564208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:55:25.564208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:55:25.564208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:55:25.564208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:55:25.564208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:55:25.564208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:25.564208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:25.564208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 528/1043] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:55:25 2022] GET /identity/v3/projects/service => generated 88 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:25.564702 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb2717-e2bc-40f1-ab7f-a4d326fda4e2 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:25.594754 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:25.670151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbfb2717-e2bc-40f1-ab7f-a4d326fda4e2 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:25.705182 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9276624d-f3d6-40e1-bf03-15ec0eed1886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:25.705847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276624d-f3d6-40e1-bf03-15ec0eed1886 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:25.706137 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276624d-f3d6-40e1-bf03-15ec0eed1886 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:25.706410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9276624d-f3d6-40e1-bf03-15ec0eed1886 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:25.707203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276624d-f3d6-40e1-bf03-15ec0eed1886 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:25.707522 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276624d-f3d6-40e1-bf03-15ec0eed1886 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:25.709007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9276624d-f3d6-40e1-bf03-15ec0eed1886 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:25.721310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 516/1044] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:55:25 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:25.727135 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3abac-564d-4a77-a12e-9892d1ccef6a None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:25.761677 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:25.828161 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c3abac-564d-4a77-a12e-9892d1ccef6a None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:25.862542 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88ca2f1-53d0-4c30-8313-d9e84a150a80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:25.863199 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ca2f1-53d0-4c30-8313-d9e84a150a80 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:25.863488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ca2f1-53d0-4c30-8313-d9e84a150a80 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:25.863773 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88ca2f1-53d0-4c30-8313-d9e84a150a80 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:25.864640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ca2f1-53d0-4c30-8313-d9e84a150a80 None admin] RBAC: Adding query filter params (role.id=f1cd5d80f39b48bdb388dd5c539d9a0b, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=4751abc9f67947b3ad17ba2b78ce2c12) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:25.864964 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ca2f1-53d0-4c30-8313-d9e84a150a80 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:25.866418 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88ca2f1-53d0-4c30-8313-d9e84a150a80 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:25.872163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 529/1045] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:55:25 2022] GET /identity/v3/role_assignments?user.id=4751abc9f67947b3ad17ba2b78ce2c12&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=f1cd5d80f39b48bdb388dd5c539d9a0b => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:27.020016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b37850-a5d3-44a9-a9ee-d7263758904f None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:27.020545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b37850-a5d3-44a9-a9ee-d7263758904f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:27.020683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97b37850-a5d3-44a9-a9ee-d7263758904f None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:27.021973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 517/1046] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:27.027682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec8daf-984a-4f17-a955-26b2b9a89a24 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:27.028133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec8daf-984a-4f17-a955-26b2b9a89a24 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:27.028302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec8daf-984a-4f17-a955-26b2b9a89a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:27.046740 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-deec8daf-984a-4f17-a955-26b2b9a89a24 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:27.047993 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-deec8daf-984a-4f17-a955-26b2b9a89a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:27.107095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09621804-49a5-4f94-ac0f-58721da21ebb None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:27.107495 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09621804-49a5-4f94-ac0f-58721da21ebb None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:27.107800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09621804-49a5-4f94-ac0f-58721da21ebb None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:27.109247 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 518/1047] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:27.114908 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dafae5-86f5-4e38-921f-40a9a021ba5e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:27.115205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dafae5-86f5-4e38-921f-40a9a021ba5e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:27.115491 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dafae5-86f5-4e38-921f-40a9a021ba5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:27.124085 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 530/1048] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:27.132113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-61dafae5-86f5-4e38-921f-40a9a021ba5e None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:27.133322 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-61dafae5-86f5-4e38-921f-40a9a021ba5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:27.142909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec8daf-984a-4f17-a955-26b2b9a89a24 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:27.173113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:27.203233 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 519/1049] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:27 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:27.226897 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dafae5-86f5-4e38-921f-40a9a021ba5e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:27.245890 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:27.261436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec8daf-984a-4f17-a955-26b2b9a89a24 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:27.305515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d28862-a303-4c7e-a924-daf19ba77e85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:27.306153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d28862-a303-4c7e-a924-daf19ba77e85 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:27.306506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d28862-a303-4c7e-a924-daf19ba77e85 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:27.306789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d28862-a303-4c7e-a924-daf19ba77e85 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:27.317907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d28862-a303-4c7e-a924-daf19ba77e85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:55:27.325803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-85d28862-a303-4c7e-a924-daf19ba77e85 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:55:27.325803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:27.325803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:27.325803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:27.325803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:27.325803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:27.325803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:27.325803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:27.325803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:27.325803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:27.325803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:27.325803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:27.325803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:55:27.325803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:55:27.325803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:55:27.325803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:55:27.326748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:55:27.326748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:55:27.326748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:55:27.326748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:55:27.326748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:27.326748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:27.326748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:27.326748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:27.326748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:27.326748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:27.326748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:27.326748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:27.326748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:27.326748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:27.326748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:27.326748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:27.327594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:27.327594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:27.327594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:27.327594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:27.327594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:55:27.327594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:55:27.327594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:55:27.327594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:55:27.327594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:55:27.327594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:55:27.327594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:55:27.327594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:27.327594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 531/1050] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:55:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:55:27.333265 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d28862-a303-4c7e-a924-daf19ba77e85 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:27.339537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dafae5-86f5-4e38-921f-40a9a021ba5e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:27.377626 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc2dc4-3634-46b9-a8e6-a5ab97defaf0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:27.378489 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc2dc4-3634-46b9-a8e6-a5ab97defaf0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:27.378782 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc2dc4-3634-46b9-a8e6-a5ab97defaf0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:27.379066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aebc2dc4-3634-46b9-a8e6-a5ab97defaf0 None admin] PATH_INFO: `/v3/services/image` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:27.379734 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc2dc4-3634-46b9-a8e6-a5ab97defaf0 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:27.381171 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aebc2dc4-3634-46b9-a8e6-a5ab97defaf0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:27.393536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-aebc2dc4-3634-46b9-a8e6-a5ab97defaf0 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. May 25 15:55:27.393536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:27.393536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:55:27.393536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:55:27.393536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:55:27.393536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:55:27.393536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:55:27.393536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:27.393536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 15:55:27.393536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:27.393536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:55:27.393536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:27.393536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:27.393536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:27.393536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:27.393536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:27.393536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:27.394571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:27.394571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:27.394571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:27.394571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:27.394571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:27.394571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:27.394571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:55:27.394571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:55:27.394571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:55:27.394571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:55:27.394571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:27.394571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:27.394571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:27.394571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:27.394571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:27.394571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:27.396658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:27.396658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:27.396658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:27.396658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:27.396658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:27.396658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:27.396658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:27.396658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:27.396658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:27.396658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:27.396658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:55:27.396658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:55:27.396658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. May 25 15:55:27.396658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:27.396658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 520/1051] 10.222.0.85 () {56 vars in 1105 bytes} [Wed May 25 15:55:27 2022] GET /identity/v3/services/image => generated 86 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:55:27.400803 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc2dc4-3634-46b9-a8e6-a5ab97defaf0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:27.435598 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:27.448738 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d28862-a303-4c7e-a924-daf19ba77e85 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:27.482517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d2fe4-41f4-4af3-8d13-0ee5ff03be73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:27.483201 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d2fe4-41f4-4af3-8d13-0ee5ff03be73 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:27.483493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d2fe4-41f4-4af3-8d13-0ee5ff03be73 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:27.483776 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-327d2fe4-41f4-4af3-8d13-0ee5ff03be73 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:27.484568 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d2fe4-41f4-4af3-8d13-0ee5ff03be73 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:27.484885 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d2fe4-41f4-4af3-8d13-0ee5ff03be73 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:27.486281 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-327d2fe4-41f4-4af3-8d13-0ee5ff03be73 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:27.488376 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:27.495433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 532/1052] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:55:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:27.501046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d2fe4-41f4-4af3-8d13-0ee5ff03be73 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:27.510573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aebc2dc4-3634-46b9-a8e6-a5ab97defaf0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:27.547682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3978c9-5b26-4489-83b2-d36887ee7b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:27.548405 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3978c9-5b26-4489-83b2-d36887ee7b25 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:27.548667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3978c9-5b26-4489-83b2-d36887ee7b25 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:27.548947 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c3978c9-5b26-4489-83b2-d36887ee7b25 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:27.549656 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3978c9-5b26-4489-83b2-d36887ee7b25 None admin] RBAC: Adding query filter params (name=image) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:27.549966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3978c9-5b26-4489-83b2-d36887ee7b25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:27.551350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c3978c9-5b26-4489-83b2-d36887ee7b25 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:27.560988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 521/1053] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:27 2022] GET /identity/v3/services?name=image => generated 122 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:27.562223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3978c9-5b26-4489-83b2-d36887ee7b25 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:27.607640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-327d2fe4-41f4-4af3-8d13-0ee5ff03be73 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:27.645966 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2643e69-20b8-4070-978b-1dc1ce36374c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:27.646869 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2643e69-20b8-4070-978b-1dc1ce36374c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:27.647163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2643e69-20b8-4070-978b-1dc1ce36374c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:27.647450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2643e69-20b8-4070-978b-1dc1ce36374c None admin] PATH_INFO: `/v3/users/06da52bc45d5424697b407bc5deda3bd` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:27.663081 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2643e69-20b8-4070-978b-1dc1ce36374c None admin] RBAC: Authorizing `identity:get_user(user_id=06da52bc45d5424697b407bc5deda3bd)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:27.664617 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2643e69-20b8-4070-978b-1dc1ce36374c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:27.666387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 533/1054] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:55:27 2022] GET /identity/v3/users/06da52bc45d5424697b407bc5deda3bd => generated 285 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:27.668191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c3978c9-5b26-4489-83b2-d36887ee7b25 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:27.672010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2643e69-20b8-4070-978b-1dc1ce36374c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:27.682602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:27.707219 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a556ab5a-6d39-428c-9c6b-142925b2f668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:27.707976 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556ab5a-6d39-428c-9c6b-142925b2f668 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:27.708268 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556ab5a-6d39-428c-9c6b-142925b2f668 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:27.708714 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a556ab5a-6d39-428c-9c6b-142925b2f668 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:27.710215 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a556ab5a-6d39-428c-9c6b-142925b2f668 None admin] RBAC: Adding query filter params (type=image) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:27.710542 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a556ab5a-6d39-428c-9c6b-142925b2f668 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:27.712013 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a556ab5a-6d39-428c-9c6b-142925b2f668 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:27.717740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 522/1055] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:27 2022] GET /identity/v3/services?type=image => generated 350 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:27.723282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a556ab5a-6d39-428c-9c6b-142925b2f668 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:27.736756 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:27.782508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2643e69-20b8-4070-978b-1dc1ce36374c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:27.816411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14030545-b793-4813-8ffa-0540cea888c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:27.816997 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14030545-b793-4813-8ffa-0540cea888c3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:27.817290 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14030545-b793-4813-8ffa-0540cea888c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:27.817577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14030545-b793-4813-8ffa-0540cea888c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:27.831307 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a556ab5a-6d39-428c-9c6b-142925b2f668 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:27.868588 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf4154-8685-49db-82b7-f092dfdd494d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:27.869450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf4154-8685-49db-82b7-f092dfdd494d None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:27.869743 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf4154-8685-49db-82b7-f092dfdd494d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:27.870127 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cf4154-8685-49db-82b7-f092dfdd494d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:27.870925 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf4154-8685-49db-82b7-f092dfdd494d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:27.872393 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cf4154-8685-49db-82b7-f092dfdd494d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:27.891153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14030545-b793-4813-8ffa-0540cea888c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:27.892704 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14030545-b793-4813-8ffa-0540cea888c3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:27.895672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 523/1056] 10.222.0.85 () {60 vars in 1144 bytes} [Wed May 25 15:55:27 2022] POST /identity/v3/endpoints => generated 327 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:55:27.907625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:28.009130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 534/1057] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:55:27 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 15:55:28.015139 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf4154-8685-49db-82b7-f092dfdd494d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:28.060056 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3103f-d3ef-44b3-a916-4940f9bc4b8f None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:28.060618 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3103f-d3ef-44b3-a916-4940f9bc4b8f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:28.060618 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3103f-d3ef-44b3-a916-4940f9bc4b8f None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:28.062178 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 535/1058] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:28.069174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc2f37-0175-4edd-a9fd-9d0afe1c7f0f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:28.069384 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc2f37-0175-4edd-a9fd-9d0afe1c7f0f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:28.069643 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49fc2f37-0175-4edd-a9fd-9d0afe1c7f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:28.089009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-49fc2f37-0175-4edd-a9fd-9d0afe1c7f0f None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:28.090237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-49fc2f37-0175-4edd-a9fd-9d0afe1c7f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:28.096476 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:28.147003 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cf4154-8685-49db-82b7-f092dfdd494d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:28.165368 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 536/1059] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:55:28.181062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4f9a9-7684-4ae7-a47b-e5a2fa9f29a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:28.181971 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4f9a9-7684-4ae7-a47b-e5a2fa9f29a7 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:28.182256 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4f9a9-7684-4ae7-a47b-e5a2fa9f29a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:28.182532 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a4f9a9-7684-4ae7-a47b-e5a2fa9f29a7 None admin] PATH_INFO: `/v3/projects/c0881e3bcd064aa58e5a8187f5032cbd` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:28.183705 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4f9a9-7684-4ae7-a47b-e5a2fa9f29a7 None admin] RBAC: Authorizing `identity:get_project(project_id=c0881e3bcd064aa58e5a8187f5032cbd)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:28.185187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a4f9a9-7684-4ae7-a47b-e5a2fa9f29a7 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:28.187039 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 524/1060] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:55:28 2022] GET /identity/v3/projects/c0881e3bcd064aa58e5a8187f5032cbd => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:28.188289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc2f37-0175-4edd-a9fd-9d0afe1c7f0f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:28.192606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4f9a9-7684-4ae7-a47b-e5a2fa9f29a7 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:28.233794 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:28.298048 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49fc2f37-0175-4edd-a9fd-9d0afe1c7f0f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:28.301618 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a4f9a9-7684-4ae7-a47b-e5a2fa9f29a7 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:28.330478 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:28.336461 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b367be-f50f-4d9a-83bb-d5365c5eee6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:28.337126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b367be-f50f-4d9a-83bb-d5365c5eee6c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:28.337433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b367be-f50f-4d9a-83bb-d5365c5eee6c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:28.337712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9b367be-f50f-4d9a-83bb-d5365c5eee6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:28.338609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b367be-f50f-4d9a-83bb-d5365c5eee6c None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=c0881e3bcd064aa58e5a8187f5032cbd, user.id=06da52bc45d5424697b407bc5deda3bd) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:28.338950 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b367be-f50f-4d9a-83bb-d5365c5eee6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:28.340476 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9b367be-f50f-4d9a-83bb-d5365c5eee6c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:28.340987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe679d-a3d7-41c9-b696-04b36cf069e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:28.341810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe679d-a3d7-41c9-b696-04b36cf069e1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:28.342240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe679d-a3d7-41c9-b696-04b36cf069e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:28.342538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cfe679d-a3d7-41c9-b696-04b36cf069e1 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:28.346090 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 525/1061] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:55:28 2022] GET /identity/v3/role_assignments?user.id=06da52bc45d5424697b407bc5deda3bd&scope.project.id=c0881e3bcd064aa58e5a8187f5032cbd&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:28.354371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cfe679d-a3d7-41c9-b696-04b36cf069e1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:55:28.362960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2cfe679d-a3d7-41c9-b696-04b36cf069e1 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 15:55:28.362960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:28.362960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:28.362960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:28.362960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:28.362960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:28.362960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:28.362960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:28.362960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:28.362960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:28.362960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:28.362960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:28.362960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:55:28.362960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:55:28.362960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:55:28.362960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:55:28.364030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:55:28.364030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:55:28.364030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:55:28.364030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:55:28.364030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:28.364030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:28.364030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:28.364030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:28.364030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:28.364030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:28.364030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:28.364030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:28.364030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:28.364030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:28.364030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:28.364030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:28.364913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:28.364913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:28.364913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:28.364913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:28.364913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:55:28.364913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:55:28.364913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:55:28.364913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:55:28.364913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:55:28.364913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:55:28.364913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 15:55:28.364913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:28.368251 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 537/1062] 10.222.0.85 () {56 vars in 1120 bytes} [Wed May 25 15:55:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:28.370647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b367be-f50f-4d9a-83bb-d5365c5eee6c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:28.484154 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9b367be-f50f-4d9a-83bb-d5365c5eee6c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:28.521296 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d9468-d4f7-4775-8bfd-fcc973b84f94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:28.521993 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d9468-d4f7-4775-8bfd-fcc973b84f94 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:28.522279 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d9468-d4f7-4775-8bfd-fcc973b84f94 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:28.522562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9d9468-d4f7-4775-8bfd-fcc973b84f94 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:28.523326 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d9468-d4f7-4775-8bfd-fcc973b84f94 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:28.523763 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d9468-d4f7-4775-8bfd-fcc973b84f94 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:28.525283 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e9d9468-d4f7-4775-8bfd-fcc973b84f94 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:28.534665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 526/1063] 10.222.0.85 () {56 vars in 1115 bytes} [Wed May 25 15:55:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:28.540599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe679d-a3d7-41c9-b696-04b36cf069e1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:28.642000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:28.646242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cfe679d-a3d7-41c9-b696-04b36cf069e1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:28.682364 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef3f52-58aa-4600-ae18-4a142099566d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:28.683175 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef3f52-58aa-4600-ae18-4a142099566d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:28.683468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef3f52-58aa-4600-ae18-4a142099566d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:28.683752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ef3f52-58aa-4600-ae18-4a142099566d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:28.695035 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef3f52-58aa-4600-ae18-4a142099566d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:28.696547 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ef3f52-58aa-4600-ae18-4a142099566d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:28.714412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-d0ef3f52-58aa-4600-ae18-4a142099566d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:28.714412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:28.714412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:28.714412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:28.714412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:28.714412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:28.714412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:28.714412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:28.714412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:28.714412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:28.714412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:28.714412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:28.714412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:55:28.714412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:55:28.714412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:55:28.715411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:55:28.715411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:28.715411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:28.715411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:28.715411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:28.715411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:28.715411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:28.715411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:28.715411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:28.715411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:28.715411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:28.715411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:28.715411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:28.715411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:28.715411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:28.715411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:28.715411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:28.716362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:55:28.716362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:55:28.716362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:28.716362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:28.719900 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 538/1064] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:28.721504 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d9468-d4f7-4775-8bfd-fcc973b84f94 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:28.779597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:28.826314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e9d9468-d4f7-4775-8bfd-fcc973b84f94 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:28.860228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb3a45-b4fa-4cc2-bf05-5e4be46f5578 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:28.860896 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb3a45-b4fa-4cc2-bf05-5e4be46f5578 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:28.861187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb3a45-b4fa-4cc2-bf05-5e4be46f5578 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:28.861470 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbeb3a45-b4fa-4cc2-bf05-5e4be46f5578 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:28.862203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb3a45-b4fa-4cc2-bf05-5e4be46f5578 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:28.862500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb3a45-b4fa-4cc2-bf05-5e4be46f5578 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:28.863918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbeb3a45-b4fa-4cc2-bf05-5e4be46f5578 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:28.876034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 527/1065] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:55:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:28.881573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef3f52-58aa-4600-ae18-4a142099566d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:28.983935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ef3f52-58aa-4600-ae18-4a142099566d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:29.017613 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77be8c-f76a-410d-82cb-e7ce850211ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:29.018442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77be8c-f76a-410d-82cb-e7ce850211ab None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:29.018730 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77be8c-f76a-410d-82cb-e7ce850211ab None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:29.019013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e77be8c-f76a-410d-82cb-e7ce850211ab None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:29.029083 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77be8c-f76a-410d-82cb-e7ce850211ab None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:29.030628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e77be8c-f76a-410d-82cb-e7ce850211ab None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:29.046123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2e77be8c-f76a-410d-82cb-e7ce850211ab None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 15:55:29.046123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:29.046123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:29.046123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:29.046123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:29.046123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:29.046123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:29.046123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:29.046123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:29.046123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:29.046123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:29.046123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:29.046123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:55:29.046123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:55:29.046123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:55:29.046123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:29.047173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:29.048566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:29.048566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:29.048566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:55:29.048566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:55:29.048566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:55:29.048566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:55:29.048566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:55:29.048566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:55:29.048566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 15:55:29.048566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:29.049459 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 539/1066] 10.222.0.85 () {56 vars in 1093 bytes} [Wed May 25 15:55:28 2022] GET /identity/v3/users/nova => generated 82 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:29.053192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb3a45-b4fa-4cc2-bf05-5e4be46f5578 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:29.160104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbeb3a45-b4fa-4cc2-bf05-5e4be46f5578 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:29.177419 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:29.196672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6dbe5-324a-4070-98d2-c9aebb8da034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:29.197387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6dbe5-324a-4070-98d2-c9aebb8da034 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:29.197675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6dbe5-324a-4070-98d2-c9aebb8da034 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:29.197956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce6dbe5-324a-4070-98d2-c9aebb8da034 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:29.198806 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6dbe5-324a-4070-98d2-c9aebb8da034 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:29.199122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6dbe5-324a-4070-98d2-c9aebb8da034 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:29.200591 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bce6dbe5-324a-4070-98d2-c9aebb8da034 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:29.218251 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 528/1067] 10.222.0.85 () {56 vars in 1133 bytes} [Wed May 25 15:55:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:29.223748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77be8c-f76a-410d-82cb-e7ce850211ab None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:29.237111 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:29.331957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e77be8c-f76a-410d-82cb-e7ce850211ab None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:29.369190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de11b92-4e09-4b52-9313-728a25c283bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:29.369990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de11b92-4e09-4b52-9313-728a25c283bf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:29.370278 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de11b92-4e09-4b52-9313-728a25c283bf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:29.370558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de11b92-4e09-4b52-9313-728a25c283bf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:29.381379 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de11b92-4e09-4b52-9313-728a25c283bf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:29.382839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de11b92-4e09-4b52-9313-728a25c283bf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:29.399780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-9de11b92-4e09-4b52-9313-728a25c283bf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:29.399780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:29.399780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:29.399780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:29.399780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:29.399780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:29.399780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:29.399780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:29.399780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:29.399780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:29.399780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:29.399780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:29.399780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:55:29.399780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:55:29.399780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:55:29.400883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:55:29.400883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:29.400883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:29.400883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:29.400883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:29.400883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:29.400883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:29.400883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:29.400883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:29.400883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:29.400883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:29.400883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:29.400883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:29.400883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:29.400883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:29.400883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:29.400883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:29.401906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:55:29.401906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:55:29.401906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:29.401906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:29.403915 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 540/1068] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:29.406818 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6dbe5-324a-4070-98d2-c9aebb8da034 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:29.521381 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce6dbe5-324a-4070-98d2-c9aebb8da034 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:29.556745 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e56899-ba22-4c20-8824-b797749350db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:29.557417 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e56899-ba22-4c20-8824-b797749350db None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:29.557704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e56899-ba22-4c20-8824-b797749350db None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:29.557986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e56899-ba22-4c20-8824-b797749350db None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:29.558709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e56899-ba22-4c20-8824-b797749350db None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:29.559022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e56899-ba22-4c20-8824-b797749350db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:29.560439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e56899-ba22-4c20-8824-b797749350db None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:29.572085 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 529/1069] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:55:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:29.577765 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de11b92-4e09-4b52-9313-728a25c283bf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:29.643533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:29.687402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de11b92-4e09-4b52-9313-728a25c283bf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:29.726752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0112538-fe0e-4046-bb84-4b9c83b4bbf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:29.727604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0112538-fe0e-4046-bb84-4b9c83b4bbf3 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:29.727973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0112538-fe0e-4046-bb84-4b9c83b4bbf3 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:29.728261 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0112538-fe0e-4046-bb84-4b9c83b4bbf3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:29.735183 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0112538-fe0e-4046-bb84-4b9c83b4bbf3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:29.736677 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0112538-fe0e-4046-bb84-4b9c83b4bbf3 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:29.751303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c0112538-fe0e-4046-bb84-4b9c83b4bbf3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:29.751303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:29.751303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:29.751303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:29.751303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:29.751303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:29.751303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:29.751303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:29.751303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:29.751303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:29.751303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:29.751303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:29.751303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:55:29.751303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:55:29.751303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:55:29.752427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:55:29.752427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:29.752427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:29.752427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:29.752427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:29.752427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:29.752427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:29.752427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:29.752427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:29.752427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:29.752427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:29.752427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:29.752427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:29.752427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:29.752427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:29.752427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:29.752427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:29.758216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:55:29.758216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:55:29.758216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:55:29.758216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:55:29.758216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:55:29.758216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:55:29.758216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:29.758216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:29.758216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 541/1070] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:55:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:29.760383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e56899-ba22-4c20-8824-b797749350db None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:29.796035 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:29.863332 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e56899-ba22-4c20-8824-b797749350db None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:29.898126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f508aa30-8c1c-4d50-8f86-8dbe597e0be6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:29.898796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508aa30-8c1c-4d50-8f86-8dbe597e0be6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:29.899086 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508aa30-8c1c-4d50-8f86-8dbe597e0be6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:29.899368 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f508aa30-8c1c-4d50-8f86-8dbe597e0be6 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:29.900218 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508aa30-8c1c-4d50-8f86-8dbe597e0be6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:29.900536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508aa30-8c1c-4d50-8f86-8dbe597e0be6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:29.901980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f508aa30-8c1c-4d50-8f86-8dbe597e0be6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:29.918931 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 530/1071] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:55:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:29.920650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0112538-fe0e-4046-bb84-4b9c83b4bbf3 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:30.032413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0112538-fe0e-4046-bb84-4b9c83b4bbf3 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:30.056031 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:30.069368 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cce3b2f4-6557-458b-a9cc-0a6a27dd86bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:30.070284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3b2f4-6557-458b-a9cc-0a6a27dd86bc None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:30.070566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3b2f4-6557-458b-a9cc-0a6a27dd86bc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:30.070844 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cce3b2f4-6557-458b-a9cc-0a6a27dd86bc None admin] PATH_INFO: `/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/4751abc9f67947b3ad17ba2b78ce2c12/roles/f1cd5d80f39b48bdb388dd5c539d9a0b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:30.073114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3b2f4-6557-458b-a9cc-0a6a27dd86bc None admin] RBAC: Authorizing `identity:create_grant(project_id=5584ebdaf133444ca69efc5554bf48e9, user_id=4751abc9f67947b3ad17ba2b78ce2c12, role_id=f1cd5d80f39b48bdb388dd5c539d9a0b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:30.074593 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cce3b2f4-6557-458b-a9cc-0a6a27dd86bc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:30.089880 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 542/1072] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:55:29 2022] PUT /identity/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/4751abc9f67947b3ad17ba2b78ce2c12/roles/f1cd5d80f39b48bdb388dd5c539d9a0b => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:55:30.138716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b8262-1213-4191-b6e8-33dceebcd143 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:30.139060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b8262-1213-4191-b6e8-33dceebcd143 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:30.139362 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e6b8262-1213-4191-b6e8-33dceebcd143 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:30.141064 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 531/1073] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:30 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:30.146938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240d427-1654-4eff-b9d2-277f2ccf934f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:30.147285 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240d427-1654-4eff-b9d2-277f2ccf934f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:30.147525 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1240d427-1654-4eff-b9d2-277f2ccf934f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:30.166552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1240d427-1654-4eff-b9d2-277f2ccf934f None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:30.167785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1240d427-1654-4eff-b9d2-277f2ccf934f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:30.242851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 543/1074] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:30.265477 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b8262-1213-4191-b6e8-33dceebcd143 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:30.383702 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:30.394044 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e6b8262-1213-4191-b6e8-33dceebcd143 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:30.432013 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f86607-18e4-4ebb-8dc2-41e79cd158c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:30.432722 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f86607-18e4-4ebb-8dc2-41e79cd158c6 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:30.433014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f86607-18e4-4ebb-8dc2-41e79cd158c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:30.433301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f86607-18e4-4ebb-8dc2-41e79cd158c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:30.501210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f86607-18e4-4ebb-8dc2-41e79cd158c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:30.502678 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f86607-18e4-4ebb-8dc2-41e79cd158c6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:30.504807 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7f2cb-3f2e-4540-8ebd-1f62e9ff5514 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:30.505100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7f2cb-3f2e-4540-8ebd-1f62e9ff5514 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:30.505414 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c7f2cb-3f2e-4540-8ebd-1f62e9ff5514 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:30.506795 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 544/1075] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:30.511761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6ef31-be0a-4533-9ef1-064d69490266 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:30.512141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6ef31-be0a-4533-9ef1-064d69490266 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:30.512493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81c6ef31-be0a-4533-9ef1-064d69490266 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:30.529816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-81c6ef31-be0a-4533-9ef1-064d69490266 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:30.531019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-81c6ef31-be0a-4533-9ef1-064d69490266 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:30.600360 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:30.608697 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 545/1076] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:30 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:55:30.614543 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 532/1077] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:55:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:55:30.615932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6ef31-be0a-4533-9ef1-064d69490266 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:30.630919 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f86607-18e4-4ebb-8dc2-41e79cd158c6 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:30.665747 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:30.727750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81c6ef31-be0a-4533-9ef1-064d69490266 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:30.731352 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f86607-18e4-4ebb-8dc2-41e79cd158c6 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:30.763575 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b13e5d-352b-4124-a64f-0d5d2428c279 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:30.764754 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b13e5d-352b-4124-a64f-0d5d2428c279 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:30.765351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b13e5d-352b-4124-a64f-0d5d2428c279 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:30.765899 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b13e5d-352b-4124-a64f-0d5d2428c279 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:30.766850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe61a24-b5ac-4903-ab0f-5041be0dfb4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:30.767968 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe61a24-b5ac-4903-ab0f-5041be0dfb4e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:30.768267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe61a24-b5ac-4903-ab0f-5041be0dfb4e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:30.768982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe61a24-b5ac-4903-ab0f-5041be0dfb4e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:30.776595 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b13e5d-352b-4124-a64f-0d5d2428c279 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:30.778069 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b13e5d-352b-4124-a64f-0d5d2428c279 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:30.781673 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe61a24-b5ac-4903-ab0f-5041be0dfb4e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:55:30.790757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-2fe61a24-b5ac-4903-ab0f-5041be0dfb4e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:55:30.790757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:30.790757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:30.790757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:30.790757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:30.790757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:30.790757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:30.790757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:30.790757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:30.790757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:30.790757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:30.790757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:30.790757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:55:30.790757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:55:30.790757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:55:30.790757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:55:30.791700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:55:30.791700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:55:30.791700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:55:30.791700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:55:30.791700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:30.791700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:30.791700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:30.791700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:30.791700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:30.791700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:30.791700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:30.791700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:30.791700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:30.791700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:30.791700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:30.791700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:30.797891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:30.797891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:30.797891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:30.797891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:30.797891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:55:30.797891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:55:30.797891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:55:30.797891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:55:30.797891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:55:30.797891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:55:30.797891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:55:30.797891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:30.797891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 533/1078] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:55:30 2022] GET /identity/v3/roles/admin => generated 83 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:55:30.799117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e1b13e5d-352b-4124-a64f-0d5d2428c279 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:30.799117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:30.799117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:30.799117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:30.799117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:30.799117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:30.799117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:30.799117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:30.799117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:30.799117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:30.799117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:30.799117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:30.799117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:55:30.799117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:55:30.799117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:55:30.800153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:55:30.800153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:30.800153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:30.800153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:30.800153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:30.800153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:30.800153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:30.800153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:30.800153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:30.800153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:30.800153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:30.800153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:30.800153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:30.800153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:30.800153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:30.800153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:30.800153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:30.801378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:55:30.801378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:55:30.801378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:30.801378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:30.802215 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 546/1079] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:55:30.802820 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe61a24-b5ac-4903-ab0f-5041be0dfb4e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:30.803132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b13e5d-352b-4124-a64f-0d5d2428c279 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:30.862277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:30.904065 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:30.909871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe61a24-b5ac-4903-ab0f-5041be0dfb4e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:30.912241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b13e5d-352b-4124-a64f-0d5d2428c279 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:30.944946 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6665b4-8aa5-456b-bc78-7bda98cfc55e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:30.945899 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6665b4-8aa5-456b-bc78-7bda98cfc55e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:30.946192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6665b4-8aa5-456b-bc78-7bda98cfc55e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:30.946473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d6665b4-8aa5-456b-bc78-7bda98cfc55e None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:30.947289 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6665b4-8aa5-456b-bc78-7bda98cfc55e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:30.947719 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6665b4-8aa5-456b-bc78-7bda98cfc55e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:30.948296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653390ed-5a1f-4ceb-a229-3ba40a553381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:30.948966 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653390ed-5a1f-4ceb-a229-3ba40a553381 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:30.949119 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d6665b4-8aa5-456b-bc78-7bda98cfc55e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:30.949291 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653390ed-5a1f-4ceb-a229-3ba40a553381 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:30.949591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653390ed-5a1f-4ceb-a229-3ba40a553381 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:30.950466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653390ed-5a1f-4ceb-a229-3ba40a553381 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:30.950784 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653390ed-5a1f-4ceb-a229-3ba40a553381 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:30.952339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653390ed-5a1f-4ceb-a229-3ba40a553381 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:30.957864 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 534/1080] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:55:30 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:30.963619 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6665b4-8aa5-456b-bc78-7bda98cfc55e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:30.964437 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 547/1081] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:55:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:31.076278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d6665b4-8aa5-456b-bc78-7bda98cfc55e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:31.094793 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:31.116744 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c34f2b-9502-44ab-8b92-cbcaa7a43f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:31.117724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c34f2b-9502-44ab-8b92-cbcaa7a43f91 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:31.118015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c34f2b-9502-44ab-8b92-cbcaa7a43f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:31.118295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19c34f2b-9502-44ab-8b92-cbcaa7a43f91 None admin] PATH_INFO: `/v3/users/06da52bc45d5424697b407bc5deda3bd` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:31.119667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c34f2b-9502-44ab-8b92-cbcaa7a43f91 None admin] RBAC: Authorizing `identity:get_user(user_id=06da52bc45d5424697b407bc5deda3bd)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:31.121214 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19c34f2b-9502-44ab-8b92-cbcaa7a43f91 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:31.123107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 535/1082] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:55:30 2022] GET /identity/v3/users/06da52bc45d5424697b407bc5deda3bd => generated 285 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:31.131859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653390ed-5a1f-4ceb-a229-3ba40a553381 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:31.240548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653390ed-5a1f-4ceb-a229-3ba40a553381 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:31.277081 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1274dbe-c9a7-44e4-ba8f-b3897c43c142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:31.278203 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1274dbe-c9a7-44e4-ba8f-b3897c43c142 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:31.278310 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1274dbe-c9a7-44e4-ba8f-b3897c43c142 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:31.278508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1274dbe-c9a7-44e4-ba8f-b3897c43c142 None admin] PATH_INFO: `/v3/projects/c0881e3bcd064aa58e5a8187f5032cbd` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:31.279691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1274dbe-c9a7-44e4-ba8f-b3897c43c142 None admin] RBAC: Authorizing `identity:get_project(project_id=c0881e3bcd064aa58e5a8187f5032cbd)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:31.281685 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1274dbe-c9a7-44e4-ba8f-b3897c43c142 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:31.283020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 548/1083] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:55:31 2022] GET /identity/v3/projects/c0881e3bcd064aa58e5a8187f5032cbd => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:31.288839 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c34f2b-9502-44ab-8b92-cbcaa7a43f91 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:31.391018 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19c34f2b-9502-44ab-8b92-cbcaa7a43f91 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:31.435961 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcb7b0e-0655-4de2-aa29-c94bc9fd530a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:31.436880 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb7b0e-0655-4de2-aa29-c94bc9fd530a None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:31.437169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb7b0e-0655-4de2-aa29-c94bc9fd530a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:31.437450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcb7b0e-0655-4de2-aa29-c94bc9fd530a None admin] PATH_INFO: `/v3/projects/c0881e3bcd064aa58e5a8187f5032cbd/users/06da52bc45d5424697b407bc5deda3bd/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:31.439684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb7b0e-0655-4de2-aa29-c94bc9fd530a None admin] RBAC: Authorizing `identity:create_grant(project_id=c0881e3bcd064aa58e5a8187f5032cbd, user_id=06da52bc45d5424697b407bc5deda3bd, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:31.441199 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcb7b0e-0655-4de2-aa29-c94bc9fd530a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:31.458302 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 536/1084] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:55:31 2022] PUT /identity/v3/projects/c0881e3bcd064aa58e5a8187f5032cbd/users/06da52bc45d5424697b407bc5deda3bd/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:55:32.282324 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc67f7-7618-4f6b-8288-91519021a55f None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:32.283001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc67f7-7618-4f6b-8288-91519021a55f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:32.283001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7fc67f7-7618-4f6b-8288-91519021a55f None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:32.284292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 549/1085] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:32.289636 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8ae53-0a14-4f17-9e0d-a46c306f92e7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:32.290071 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8ae53-0a14-4f17-9e0d-a46c306f92e7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:32.290331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c8ae53-0a14-4f17-9e0d-a46c306f92e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:32.307868 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b5c8ae53-0a14-4f17-9e0d-a46c306f92e7 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:32.309090 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b5c8ae53-0a14-4f17-9e0d-a46c306f92e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:32.351951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:32.380179 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 537/1086] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:32.403773 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc67f7-7618-4f6b-8288-91519021a55f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:32.429627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:32.513640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7fc67f7-7618-4f6b-8288-91519021a55f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:32.550668 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58156d9c-910a-45d6-b6c5-8c8580bce6d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:32.551406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58156d9c-910a-45d6-b6c5-8c8580bce6d7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:32.551697 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58156d9c-910a-45d6-b6c5-8c8580bce6d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:32.552013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58156d9c-910a-45d6-b6c5-8c8580bce6d7 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:32.562621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58156d9c-910a-45d6-b6c5-8c8580bce6d7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:55:32.571022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-58156d9c-910a-45d6-b6c5-8c8580bce6d7 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 15:55:32.571022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:32.571022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:32.571022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:32.571022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:32.571022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:32.571022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:32.571022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:32.571022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:32.571022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:32.571022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:32.571022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:32.571022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:55:32.571022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:55:32.571022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:55:32.571022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:55:32.572124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:55:32.572124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:55:32.572124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:55:32.572124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:55:32.572124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:32.572124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:32.572124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:32.572124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:32.572124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:32.572124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:32.572124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:32.572124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:32.572124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:32.572124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:32.572124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:32.572124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:32.573080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:32.573080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:32.573080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:32.573080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:32.573080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:55:32.573080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:55:32.573080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:55:32.573080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:55:32.573080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:55:32.573080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:55:32.573080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. May 25 15:55:32.573080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:32.573080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 550/1087] 10.222.0.85 () {56 vars in 1120 bytes} [Wed May 25 15:55:32 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:32.578699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8ae53-0a14-4f17-9e0d-a46c306f92e7 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:32.676484 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c8ae53-0a14-4f17-9e0d-a46c306f92e7 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:32.710212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d72f9-735d-4cb2-9fd1-bfa7a90c4207 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:32.710883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d72f9-735d-4cb2-9fd1-bfa7a90c4207 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:32.711169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d72f9-735d-4cb2-9fd1-bfa7a90c4207 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:32.711450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10d72f9-735d-4cb2-9fd1-bfa7a90c4207 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:32.712267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d72f9-735d-4cb2-9fd1-bfa7a90c4207 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:32.712577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d72f9-735d-4cb2-9fd1-bfa7a90c4207 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:32.713959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10d72f9-735d-4cb2-9fd1-bfa7a90c4207 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:32.722041 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 538/1088] 10.222.0.85 () {56 vars in 1115 bytes} [Wed May 25 15:55:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:32.728053 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58156d9c-910a-45d6-b6c5-8c8580bce6d7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:32.811512 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:32.830877 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58156d9c-910a-45d6-b6c5-8c8580bce6d7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:32.865288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd8a9e-4884-4dce-9b5e-03f528097dbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:32.866089 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd8a9e-4884-4dce-9b5e-03f528097dbb None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:32.866376 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd8a9e-4884-4dce-9b5e-03f528097dbb None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:32.866663 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04cd8a9e-4884-4dce-9b5e-03f528097dbb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:32.877639 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd8a9e-4884-4dce-9b5e-03f528097dbb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:32.879108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04cd8a9e-4884-4dce-9b5e-03f528097dbb None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:32.896386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-04cd8a9e-4884-4dce-9b5e-03f528097dbb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:32.896386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:32.896386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:32.896386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:32.896386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:32.896386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:32.896386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:32.896386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:32.896386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:32.896386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:32.896386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:32.896386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:32.896386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:55:32.896386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:55:32.896386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:55:32.897323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:55:32.897323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:32.897323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:32.897323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:32.897323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:32.897323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:32.897323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:32.897323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:32.897323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:32.897323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:32.897323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:32.897323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:32.897323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:32.897323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:32.897323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:32.897323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:32.897323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:32.898470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:55:32.898470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:55:32.898470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:32.898470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:32.898470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 551/1089] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:32.903582 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d72f9-735d-4cb2-9fd1-bfa7a90c4207 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:32.939616 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:33.005422 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10d72f9-735d-4cb2-9fd1-bfa7a90c4207 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:33.039975 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0559c7-0fe6-4970-a8e8-4499fe12d142 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:33.040672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0559c7-0fe6-4970-a8e8-4499fe12d142 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:33.040961 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0559c7-0fe6-4970-a8e8-4499fe12d142 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:33.041241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0559c7-0fe6-4970-a8e8-4499fe12d142 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:33.041964 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0559c7-0fe6-4970-a8e8-4499fe12d142 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:33.042275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0559c7-0fe6-4970-a8e8-4499fe12d142 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:33.043651 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a0559c7-0fe6-4970-a8e8-4499fe12d142 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:33.054904 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 539/1090] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:55:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:33.060791 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd8a9e-4884-4dce-9b5e-03f528097dbb None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:33.163528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04cd8a9e-4884-4dce-9b5e-03f528097dbb None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:33.210034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4cc71d-9a04-45bb-9541-4654dff1c513 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:33.211122 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cc71d-9a04-45bb-9541-4654dff1c513 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:33.211539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cc71d-9a04-45bb-9541-4654dff1c513 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:33.212208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a4cc71d-9a04-45bb-9541-4654dff1c513 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:33.214570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:33.222591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4cc71d-9a04-45bb-9541-4654dff1c513 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:33.224168 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a4cc71d-9a04-45bb-9541-4654dff1c513 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:33.239801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-5a4cc71d-9a04-45bb-9541-4654dff1c513 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. May 25 15:55:33.239801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:33.239801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:33.239801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:33.239801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:33.239801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:33.239801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:33.239801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:33.239801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:33.239801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:33.239801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:33.239801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:33.239801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:55:33.239801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:55:33.239801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:55:33.239801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:33.241334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:33.242635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:33.242635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:33.242635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:55:33.242635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:55:33.242635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:55:33.242635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:55:33.242635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:55:33.242635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:55:33.242635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. May 25 15:55:33.242635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:33.242635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 552/1091] 10.222.0.85 () {56 vars in 1093 bytes} [Wed May 25 15:55:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:33.246973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0559c7-0fe6-4970-a8e8-4499fe12d142 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:33.345894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:33.352790 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0559c7-0fe6-4970-a8e8-4499fe12d142 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:33.386651 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e613ab-8fb9-4d3a-8903-06e0df6632f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:33.387334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e613ab-8fb9-4d3a-8903-06e0df6632f8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:33.387625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e613ab-8fb9-4d3a-8903-06e0df6632f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:33.387945 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e613ab-8fb9-4d3a-8903-06e0df6632f8 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:33.388792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e613ab-8fb9-4d3a-8903-06e0df6632f8 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:33.389111 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e613ab-8fb9-4d3a-8903-06e0df6632f8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:33.390558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e613ab-8fb9-4d3a-8903-06e0df6632f8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:33.411970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 540/1092] 10.222.0.85 () {56 vars in 1133 bytes} [Wed May 25 15:55:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:33.416149 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4cc71d-9a04-45bb-9541-4654dff1c513 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:33.538388 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a4cc71d-9a04-45bb-9541-4654dff1c513 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:33.573354 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d1e44-d5ae-46ee-88ef-0f79361fda86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:33.574023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d1e44-d5ae-46ee-88ef-0f79361fda86 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:33.574315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d1e44-d5ae-46ee-88ef-0f79361fda86 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:33.574596 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3d1e44-d5ae-46ee-88ef-0f79361fda86 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:33.626456 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9274d0-bbad-4ce8-95a7-1993d6855ff3 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:33.626760 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9274d0-bbad-4ce8-95a7-1993d6855ff3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:33.627040 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9274d0-bbad-4ce8-95a7-1993d6855ff3 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:33.628622 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 541/1093] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:33.633347 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fdca7-e0b7-4944-9c4c-d3fd28002ba0 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:33.633632 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fdca7-e0b7-4944-9c4c-d3fd28002ba0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:33.633943 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-203fdca7-e0b7-4944-9c4c-d3fd28002ba0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:33.637318 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:33.645287 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d1e44-d5ae-46ee-88ef-0f79361fda86 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:33.646764 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3d1e44-d5ae-46ee-88ef-0f79361fda86 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:33.651423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-203fdca7-e0b7-4944-9c4c-d3fd28002ba0 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:33.652651 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-203fdca7-e0b7-4944-9c4c-d3fd28002ba0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:33.717939 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 542/1094] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:55:33.741001 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fdca7-e0b7-4944-9c4c-d3fd28002ba0 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:33.745315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 553/1095] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:55:33 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:55:33.750871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d1e44-d5ae-46ee-88ef-0f79361fda86 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:33.820374 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:33.836227 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-203fdca7-e0b7-4944-9c4c-d3fd28002ba0 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:33.840465 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:33.847237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3d1e44-d5ae-46ee-88ef-0f79361fda86 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:33.869628 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458abcb-d6cb-45e5-8b05-ba0e727dae09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:33.870437 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458abcb-d6cb-45e5-8b05-ba0e727dae09 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:33.870728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458abcb-d6cb-45e5-8b05-ba0e727dae09 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:33.871012 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2458abcb-d6cb-45e5-8b05-ba0e727dae09 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:33.880885 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27216e-b814-47a4-bfa2-8b50215cc218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:33.881141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2458abcb-d6cb-45e5-8b05-ba0e727dae09 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:55:33.881677 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27216e-b814-47a4-bfa2-8b50215cc218 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:33.881971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27216e-b814-47a4-bfa2-8b50215cc218 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:33.882256 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a27216e-b814-47a4-bfa2-8b50215cc218 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:33.888830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-2458abcb-d6cb-45e5-8b05-ba0e727dae09 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:55:33.888830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:33.888830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:33.888830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:33.888830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:33.888830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:33.888830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:33.888830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:33.888830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:33.888830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:33.888830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:33.888830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:33.888830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:55:33.888830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:55:33.888830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:55:33.888830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:55:33.889801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:55:33.889801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:55:33.889801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:55:33.889801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:55:33.889801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:33.889801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:33.889801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:33.889801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:33.889801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:33.889801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:33.889801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:33.889801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:33.889801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:33.889801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:33.889801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:33.889801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:33.890699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:33.890699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:33.890699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:33.890699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:33.890699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:55:33.890699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:55:33.890699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:55:33.890699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:55:33.890699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:55:33.890699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:55:33.890699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:55:33.890699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:33.890699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 543/1096] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:55:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:33.893992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27216e-b814-47a4-bfa2-8b50215cc218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:33.895525 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a27216e-b814-47a4-bfa2-8b50215cc218 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:33.896007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458abcb-d6cb-45e5-8b05-ba0e727dae09 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:33.913862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-4a27216e-b814-47a4-bfa2-8b50215cc218 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:33.913862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:33.913862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:33.913862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:33.913862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:33.913862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:33.913862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:33.913862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:33.913862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:33.913862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:33.913862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:33.913862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:33.913862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:55:33.913862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:55:33.913862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:55:33.914840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:55:33.914840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:33.914840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:33.914840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:33.914840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:33.914840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:33.914840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:33.914840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:33.914840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:33.914840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:33.914840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:33.914840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:33.914840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:33.914840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:33.914840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:33.914840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:33.914840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:33.915886 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:55:33.915886 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:55:33.915886 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:55:33.915886 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:33.915886 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 554/1097] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:55:33.920739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27216e-b814-47a4-bfa2-8b50215cc218 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:33.996264 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2458abcb-d6cb-45e5-8b05-ba0e727dae09 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:34.019493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a27216e-b814-47a4-bfa2-8b50215cc218 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:34.028444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5dfcdf-e95d-4c04-a87a-f8e0a5964e69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:34.029087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5dfcdf-e95d-4c04-a87a-f8e0a5964e69 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:34.029389 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5dfcdf-e95d-4c04-a87a-f8e0a5964e69 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:34.029665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c5dfcdf-e95d-4c04-a87a-f8e0a5964e69 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:34.030396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5dfcdf-e95d-4c04-a87a-f8e0a5964e69 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:34.030719 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5dfcdf-e95d-4c04-a87a-f8e0a5964e69 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:34.032146 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c5dfcdf-e95d-4c04-a87a-f8e0a5964e69 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:34.040061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 544/1098] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:55:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:34.045786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5dfcdf-e95d-4c04-a87a-f8e0a5964e69 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:34.054331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:34.054724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0cad0-4caa-424f-9d75-afe1b2f9e7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:34.055424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0cad0-4caa-424f-9d75-afe1b2f9e7ae None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:34.055699 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0cad0-4caa-424f-9d75-afe1b2f9e7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:34.056053 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b0cad0-4caa-424f-9d75-afe1b2f9e7ae None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:34.057193 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0cad0-4caa-424f-9d75-afe1b2f9e7ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:34.057564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0cad0-4caa-424f-9d75-afe1b2f9e7ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:34.059068 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b0cad0-4caa-424f-9d75-afe1b2f9e7ae None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:34.075032 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 555/1099] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:55:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:34.076055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0cad0-4caa-424f-9d75-afe1b2f9e7ae None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:34.084965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:34.147236 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c5dfcdf-e95d-4c04-a87a-f8e0a5964e69 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:34.174256 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b0cad0-4caa-424f-9d75-afe1b2f9e7ae None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:34.179969 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e5582-b13b-4e2e-9673-625ff1c2e74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:34.180914 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e5582-b13b-4e2e-9673-625ff1c2e74c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:34.181245 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e5582-b13b-4e2e-9673-625ff1c2e74c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:34.181525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-544e5582-b13b-4e2e-9673-625ff1c2e74c None admin] PATH_INFO: `/v3/users/06da52bc45d5424697b407bc5deda3bd` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:34.182719 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e5582-b13b-4e2e-9673-625ff1c2e74c None admin] RBAC: Authorizing `identity:get_user(user_id=06da52bc45d5424697b407bc5deda3bd)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:34.184195 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-544e5582-b13b-4e2e-9673-625ff1c2e74c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:34.185835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 545/1100] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:55:34 2022] GET /identity/v3/users/06da52bc45d5424697b407bc5deda3bd => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:34.191474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e5582-b13b-4e2e-9673-625ff1c2e74c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:34.214202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8036b0-f2f3-40fc-94f0-3ea87e9f86f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:34.215046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8036b0-f2f3-40fc-94f0-3ea87e9f86f7 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:34.215329 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8036b0-f2f3-40fc-94f0-3ea87e9f86f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:34.215614 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f8036b0-f2f3-40fc-94f0-3ea87e9f86f7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:34.222359 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8036b0-f2f3-40fc-94f0-3ea87e9f86f7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:34.223836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f8036b0-f2f3-40fc-94f0-3ea87e9f86f7 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:34.238078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8f8036b0-f2f3-40fc-94f0-3ea87e9f86f7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:34.238078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:34.238078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:34.238078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:34.238078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:34.238078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:34.238078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:34.238078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:34.238078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:34.238078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:34.238078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:34.238078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:34.238078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:55:34.238078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:55:34.238078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:55:34.239163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:55:34.239163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:34.239163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:34.239163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:34.239163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:34.239163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:34.239163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:34.239163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:34.239163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:55:34.239163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:34.239163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:34.239163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:34.239163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:34.239163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:34.239163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:34.239163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:34.239163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:34.245332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:55:34.245332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:55:34.245332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:55:34.245332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:55:34.245332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:55:34.245332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:55:34.245332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:55:34.245332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:55:34.245332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 556/1101] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:55:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:34.245332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8036b0-f2f3-40fc-94f0-3ea87e9f86f7 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:34.266118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:34.292717 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-544e5582-b13b-4e2e-9673-625ff1c2e74c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:34.312441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:34.333632 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216aae4-0a74-4e7b-bcaa-40c009085726 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:34.334300 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216aae4-0a74-4e7b-bcaa-40c009085726 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:34.334592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216aae4-0a74-4e7b-bcaa-40c009085726 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:34.334875 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a216aae4-0a74-4e7b-bcaa-40c009085726 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:34.349209 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f8036b0-f2f3-40fc-94f0-3ea87e9f86f7 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:34.383200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8497a-82c0-4e94-af6b-dcdd97e41f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:34.383953 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8497a-82c0-4e94-af6b-dcdd97e41f70 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:34.384251 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8497a-82c0-4e94-af6b-dcdd97e41f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:34.384542 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f8497a-82c0-4e94-af6b-dcdd97e41f70 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:34.385382 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8497a-82c0-4e94-af6b-dcdd97e41f70 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:34.385704 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8497a-82c0-4e94-af6b-dcdd97e41f70 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:34.387180 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f8497a-82c0-4e94-af6b-dcdd97e41f70 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:34.399236 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 557/1102] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:55:34 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:34.402903 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216aae4-0a74-4e7b-bcaa-40c009085726 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:34.404493 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a216aae4-0a74-4e7b-bcaa-40c009085726 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:34.405531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8497a-82c0-4e94-af6b-dcdd97e41f70 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:34.480423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:34.510131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 546/1103] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:55:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) May 25 15:55:34.513134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f8497a-82c0-4e94-af6b-dcdd97e41f70 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:34.517943 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216aae4-0a74-4e7b-bcaa-40c009085726 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:34.540688 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:34.552947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7180dc-7a12-4c19-a969-3ef6cb9283e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:34.553671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7180dc-7a12-4c19-a969-3ef6cb9283e8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:34.554035 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7180dc-7a12-4c19-a969-3ef6cb9283e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:34.554230 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7180dc-7a12-4c19-a969-3ef6cb9283e8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:34.555070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7180dc-7a12-4c19-a969-3ef6cb9283e8 None admin] RBAC: Adding query filter params (role.id=f1cd5d80f39b48bdb388dd5c539d9a0b, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=4751abc9f67947b3ad17ba2b78ce2c12) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:34.555395 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7180dc-7a12-4c19-a969-3ef6cb9283e8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:34.557014 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7180dc-7a12-4c19-a969-3ef6cb9283e8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:34.563128 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 558/1104] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:55:34 2022] GET /identity/v3/role_assignments?user.id=4751abc9f67947b3ad17ba2b78ce2c12&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=f1cd5d80f39b48bdb388dd5c539d9a0b => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:34.633669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a216aae4-0a74-4e7b-bcaa-40c009085726 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:34.674041 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100716dc-5a0b-4583-a6ce-a2e917a75edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:34.674943 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100716dc-5a0b-4583-a6ce-a2e917a75edd None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:34.675230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100716dc-5a0b-4583-a6ce-a2e917a75edd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:34.675512 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100716dc-5a0b-4583-a6ce-a2e917a75edd None admin] PATH_INFO: `/v3/projects/c0881e3bcd064aa58e5a8187f5032cbd` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:34.676823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100716dc-5a0b-4583-a6ce-a2e917a75edd None admin] RBAC: Authorizing `identity:get_project(project_id=c0881e3bcd064aa58e5a8187f5032cbd)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:34.678289 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100716dc-5a0b-4583-a6ce-a2e917a75edd None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:34.680089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 547/1105] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:55:34 2022] GET /identity/v3/projects/c0881e3bcd064aa58e5a8187f5032cbd => generated 311 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:34.686762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7180dc-7a12-4c19-a969-3ef6cb9283e8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:34.799937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7180dc-7a12-4c19-a969-3ef6cb9283e8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:34.836197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610974ff-4c6c-4717-9110-703efd98d922 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:34.836885 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610974ff-4c6c-4717-9110-703efd98d922 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:34.837212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610974ff-4c6c-4717-9110-703efd98d922 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:34.837518 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610974ff-4c6c-4717-9110-703efd98d922 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:34.838368 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610974ff-4c6c-4717-9110-703efd98d922 None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=c0881e3bcd064aa58e5a8187f5032cbd, user.id=06da52bc45d5424697b407bc5deda3bd) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:34.838755 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610974ff-4c6c-4717-9110-703efd98d922 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:34.840191 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610974ff-4c6c-4717-9110-703efd98d922 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:34.846430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 559/1106] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:55:34 2022] GET /identity/v3/role_assignments?user.id=06da52bc45d5424697b407bc5deda3bd&scope.project.id=c0881e3bcd064aa58e5a8187f5032cbd&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:37.050204 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fa0bf-8996-455e-b020-aff56edc2871 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:37.050775 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fa0bf-8996-455e-b020-aff56edc2871 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:37.050775 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687fa0bf-8996-455e-b020-aff56edc2871 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:37.052179 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 548/1107] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:37.057765 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de96429-c4ac-4b7e-8248-dde53d83660d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:37.058229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de96429-c4ac-4b7e-8248-dde53d83660d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:37.058402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0de96429-c4ac-4b7e-8248-dde53d83660d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:37.077330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0de96429-c4ac-4b7e-8248-dde53d83660d None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:37.078519 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0de96429-c4ac-4b7e-8248-dde53d83660d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:37.130303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:37.149663 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 560/1108] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:37.180727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fa0bf-8996-455e-b020-aff56edc2871 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:37.238082 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:37.305874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687fa0bf-8996-455e-b020-aff56edc2871 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:37.346093 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3836bdd1-fe84-4635-8322-47c66b7a8001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:37.346893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836bdd1-fe84-4635-8322-47c66b7a8001 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:37.347188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836bdd1-fe84-4635-8322-47c66b7a8001 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:37.347462 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3836bdd1-fe84-4635-8322-47c66b7a8001 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:37.348697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836bdd1-fe84-4635-8322-47c66b7a8001 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:37.350146 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3836bdd1-fe84-4635-8322-47c66b7a8001 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:37.351765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 549/1109] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:37 2022] GET /identity/v3/domains/default => generated 199 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:37.358351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de96429-c4ac-4b7e-8248-dde53d83660d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:37.483432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0de96429-c4ac-4b7e-8248-dde53d83660d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:37.523436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ce5f8f-0468-4f73-9f8b-d4cc4dd91f7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:37.524231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce5f8f-0468-4f73-9f8b-d4cc4dd91f7f None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:37.524666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce5f8f-0468-4f73-9f8b-d4cc4dd91f7f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:37.524848 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ce5f8f-0468-4f73-9f8b-d4cc4dd91f7f None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:37.525593 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce5f8f-0468-4f73-9f8b-d4cc4dd91f7f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:37.527049 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56ce5f8f-0468-4f73-9f8b-d4cc4dd91f7f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:37.555790 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 561/1110] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:55:37 2022] POST /identity/v3/users => generated 286 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:55:39.748525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f072e-9a9f-4b77-9884-356c8645ad17 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:39.749094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f072e-9a9f-4b77-9884-356c8645ad17 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:39.749094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8f072e-9a9f-4b77-9884-356c8645ad17 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:39.751191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 550/1111] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:39.760156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590b73b-8762-4917-841e-bb1b4cb80a7c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:39.760597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590b73b-8762-4917-841e-bb1b4cb80a7c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:39.760690 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8590b73b-8762-4917-841e-bb1b4cb80a7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:39.783428 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8590b73b-8762-4917-841e-bb1b4cb80a7c None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:39.784666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8590b73b-8762-4917-841e-bb1b4cb80a7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:39.821299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:39.856934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 562/1112] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:39.884811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f072e-9a9f-4b77-9884-356c8645ad17 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:39.995465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8f072e-9a9f-4b77-9884-356c8645ad17 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:40.008513 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:40.036038 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77683b33-0a31-4aec-8e0b-0322680dc4d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:40.036924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77683b33-0a31-4aec-8e0b-0322680dc4d0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:40.037214 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77683b33-0a31-4aec-8e0b-0322680dc4d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:40.037499 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77683b33-0a31-4aec-8e0b-0322680dc4d0 None admin] PATH_INFO: `/v3/roles/1cc9b5d798c648c986537d006320581f` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:40.039031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77683b33-0a31-4aec-8e0b-0322680dc4d0 None admin] RBAC: Authorizing `identity:get_role(role_id=1cc9b5d798c648c986537d006320581f)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:40.040524 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77683b33-0a31-4aec-8e0b-0322680dc4d0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:40.041976 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 551/1113] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:55:39 2022] GET /identity/v3/roles/1cc9b5d798c648c986537d006320581f => generated 223 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:40.048086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590b73b-8762-4917-841e-bb1b4cb80a7c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:40.156324 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8590b73b-8762-4917-841e-bb1b4cb80a7c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:40.202749 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122c4f3-05e5-4f61-acc3-e1a245f6fe67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:40.203999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122c4f3-05e5-4f61-acc3-e1a245f6fe67 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:40.204295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122c4f3-05e5-4f61-acc3-e1a245f6fe67 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:40.204578 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9122c4f3-05e5-4f61-acc3-e1a245f6fe67 None admin] PATH_INFO: `/v3/users/2fd321cb4fce421a938aac2fe462e71a` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:40.218402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122c4f3-05e5-4f61-acc3-e1a245f6fe67 None admin] RBAC: Authorizing `identity:get_user(user_id=2fd321cb4fce421a938aac2fe462e71a)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:40.219911 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9122c4f3-05e5-4f61-acc3-e1a245f6fe67 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:40.221543 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 563/1114] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:55:40 2022] GET /identity/v3/users/2fd321cb4fce421a938aac2fe462e71a => generated 286 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:40.229702 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77683b33-0a31-4aec-8e0b-0322680dc4d0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:40.334278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77683b33-0a31-4aec-8e0b-0322680dc4d0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:40.369065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2b1d8-ae2c-499f-9945-53f57d987aeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:40.369706 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2b1d8-ae2c-499f-9945-53f57d987aeb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:40.369993 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2b1d8-ae2c-499f-9945-53f57d987aeb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:40.370279 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a2b1d8-ae2c-499f-9945-53f57d987aeb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:40.416759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:40.439763 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2b1d8-ae2c-499f-9945-53f57d987aeb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:40.441259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a2b1d8-ae2c-499f-9945-53f57d987aeb None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:40.541167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 552/1115] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:55:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:55:40.554707 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122c4f3-05e5-4f61-acc3-e1a245f6fe67 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:40.585807 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:40.659628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9122c4f3-05e5-4f61-acc3-e1a245f6fe67 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:40.695283 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781f6de5-76a5-4134-8b79-5098f295b068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:40.696253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f6de5-76a5-4134-8b79-5098f295b068 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:40.696545 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f6de5-76a5-4134-8b79-5098f295b068 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:40.696830 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781f6de5-76a5-4134-8b79-5098f295b068 None admin] PATH_INFO: `/v3/projects/c0881e3bcd064aa58e5a8187f5032cbd` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:40.698057 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f6de5-76a5-4134-8b79-5098f295b068 None admin] RBAC: Authorizing `identity:get_project(project_id=c0881e3bcd064aa58e5a8187f5032cbd)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:40.699506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781f6de5-76a5-4134-8b79-5098f295b068 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:40.701233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 564/1116] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:55:40 2022] GET /identity/v3/projects/c0881e3bcd064aa58e5a8187f5032cbd => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:40.708038 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2b1d8-ae2c-499f-9945-53f57d987aeb None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:40.802887 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:40.827698 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a2b1d8-ae2c-499f-9945-53f57d987aeb None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:40.863359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6988df-be40-4105-a219-f78c6d52b4e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:40.864052 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6988df-be40-4105-a219-f78c6d52b4e5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:40.864336 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6988df-be40-4105-a219-f78c6d52b4e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:40.864597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6988df-be40-4105-a219-f78c6d52b4e5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:40.865452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6988df-be40-4105-a219-f78c6d52b4e5 None admin] RBAC: Adding query filter params (role.id=1cc9b5d798c648c986537d006320581f, scope.project.id=c0881e3bcd064aa58e5a8187f5032cbd, user.id=2fd321cb4fce421a938aac2fe462e71a) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:40.865772 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6988df-be40-4105-a219-f78c6d52b4e5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:40.867235 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d6988df-be40-4105-a219-f78c6d52b4e5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:40.873053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 553/1117] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:55:40 2022] GET /identity/v3/role_assignments?user.id=2fd321cb4fce421a938aac2fe462e71a&scope.project.id=c0881e3bcd064aa58e5a8187f5032cbd&role.id=1cc9b5d798c648c986537d006320581f => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:55:43.019601 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa875ea7-2b38-4de9-94d6-b5ddcbe6b561 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:43.020523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa875ea7-2b38-4de9-94d6-b5ddcbe6b561 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:43.020619 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa875ea7-2b38-4de9-94d6-b5ddcbe6b561 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:43.022689 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 565/1118] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:43.029829 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e54fe-fd72-40b1-aec0-c1ba2f9c0ec7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:43.030272 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e54fe-fd72-40b1-aec0-c1ba2f9c0ec7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:43.030411 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab4e54fe-fd72-40b1-aec0-c1ba2f9c0ec7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:43.056951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ab4e54fe-fd72-40b1-aec0-c1ba2f9c0ec7 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:43.058174 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ab4e54fe-fd72-40b1-aec0-c1ba2f9c0ec7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:43.138244 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 554/1119] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:43.161152 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa875ea7-2b38-4de9-94d6-b5ddcbe6b561 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:43.265162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:43.276055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa875ea7-2b38-4de9-94d6-b5ddcbe6b561 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:43.318345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9694729-dbaa-44f2-bae7-cd374fdd3f8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:43.319218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9694729-dbaa-44f2-bae7-cd374fdd3f8f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:43.319500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9694729-dbaa-44f2-bae7-cd374fdd3f8f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:43.319779 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9694729-dbaa-44f2-bae7-cd374fdd3f8f None admin] PATH_INFO: `/v3/roles/1cc9b5d798c648c986537d006320581f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:43.321468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9694729-dbaa-44f2-bae7-cd374fdd3f8f None admin] RBAC: Authorizing `identity:get_role(role_id=1cc9b5d798c648c986537d006320581f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:43.322858 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9694729-dbaa-44f2-bae7-cd374fdd3f8f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:43.324271 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 566/1120] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:55:43 2022] GET /identity/v3/roles/1cc9b5d798c648c986537d006320581f => generated 223 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:43.330531 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e54fe-fd72-40b1-aec0-c1ba2f9c0ec7 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:43.400524 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:43.431091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab4e54fe-fd72-40b1-aec0-c1ba2f9c0ec7 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:43.466113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d14ec-8468-403d-87a8-b6ff03768dc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:43.466996 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d14ec-8468-403d-87a8-b6ff03768dc0 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:43.467279 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d14ec-8468-403d-87a8-b6ff03768dc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:43.467563 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082d14ec-8468-403d-87a8-b6ff03768dc0 None admin] PATH_INFO: `/v3/users/2fd321cb4fce421a938aac2fe462e71a` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:43.468905 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082d14ec-8468-403d-87a8-b6ff03768dc0 None admin] RBAC: Authorizing `identity:get_user(user_id=2fd321cb4fce421a938aac2fe462e71a)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:43.470465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082d14ec-8468-403d-87a8-b6ff03768dc0 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:43.472122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 555/1121] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:55:43 2022] GET /identity/v3/users/2fd321cb4fce421a938aac2fe462e71a => generated 286 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:43.478093 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9694729-dbaa-44f2-bae7-cd374fdd3f8f None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:43.599301 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9694729-dbaa-44f2-bae7-cd374fdd3f8f None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:43.634935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e692fa-4576-4c4a-8606-ceb368ba3fd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:43.635797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e692fa-4576-4c4a-8606-ceb368ba3fd5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:43.636107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e692fa-4576-4c4a-8606-ceb368ba3fd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:43.636410 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e692fa-4576-4c4a-8606-ceb368ba3fd5 None admin] PATH_INFO: `/v3/projects/c0881e3bcd064aa58e5a8187f5032cbd` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:43.637726 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e692fa-4576-4c4a-8606-ceb368ba3fd5 None admin] RBAC: Authorizing `identity:get_project(project_id=c0881e3bcd064aa58e5a8187f5032cbd)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:43.639179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e692fa-4576-4c4a-8606-ceb368ba3fd5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:43.640903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 567/1122] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:55:43 2022] GET /identity/v3/projects/c0881e3bcd064aa58e5a8187f5032cbd => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:43.647091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d14ec-8468-403d-87a8-b6ff03768dc0 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:43.762899 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082d14ec-8468-403d-87a8-b6ff03768dc0 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:43.798230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d2cbbc-dff9-43c3-aaed-28e3eec5152d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:43.799113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2cbbc-dff9-43c3-aaed-28e3eec5152d None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:43.799400 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2cbbc-dff9-43c3-aaed-28e3eec5152d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:43.799688 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d2cbbc-dff9-43c3-aaed-28e3eec5152d None admin] PATH_INFO: `/v3/projects/c0881e3bcd064aa58e5a8187f5032cbd/users/2fd321cb4fce421a938aac2fe462e71a/roles/1cc9b5d798c648c986537d006320581f` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:43.802303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2cbbc-dff9-43c3-aaed-28e3eec5152d None admin] RBAC: Authorizing `identity:create_grant(project_id=c0881e3bcd064aa58e5a8187f5032cbd, user_id=2fd321cb4fce421a938aac2fe462e71a, role_id=1cc9b5d798c648c986537d006320581f)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:43.803765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d2cbbc-dff9-43c3-aaed-28e3eec5152d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:43.816394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:43.824624 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 556/1123] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:55:43 2022] PUT /identity/v3/projects/c0881e3bcd064aa58e5a8187f5032cbd/users/2fd321cb4fce421a938aac2fe462e71a/roles/1cc9b5d798c648c986537d006320581f => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:55:45.941093 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6187b5-4e38-4844-9a3e-0d87ab886f8f None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:45.941649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6187b5-4e38-4844-9a3e-0d87ab886f8f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:45.941649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6187b5-4e38-4844-9a3e-0d87ab886f8f None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:45.943012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 568/1124] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:45.949490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf91ea-865f-400c-8d37-58b8e616b411 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:45.950186 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf91ea-865f-400c-8d37-58b8e616b411 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:45.950341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcf91ea-865f-400c-8d37-58b8e616b411 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:45.973356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8fcf91ea-865f-400c-8d37-58b8e616b411 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:45.974650 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8fcf91ea-865f-400c-8d37-58b8e616b411 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:46.046907 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 557/1125] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:46.070758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6187b5-4e38-4844-9a3e-0d87ab886f8f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:46.105129 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:46.180760 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6187b5-4e38-4844-9a3e-0d87ab886f8f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:46.223010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef2377-a30f-4ee0-9d29-7a066411d199 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:46.223789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef2377-a30f-4ee0-9d29-7a066411d199 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:46.224106 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef2377-a30f-4ee0-9d29-7a066411d199 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:46.224385 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ef2377-a30f-4ee0-9d29-7a066411d199 None admin] PATH_INFO: `/v3/roles/1cc9b5d798c648c986537d006320581f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:46.225992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef2377-a30f-4ee0-9d29-7a066411d199 None admin] RBAC: Authorizing `identity:get_role(role_id=1cc9b5d798c648c986537d006320581f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:46.227464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ef2377-a30f-4ee0-9d29-7a066411d199 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:46.228875 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 569/1126] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:55:46 2022] GET /identity/v3/roles/1cc9b5d798c648c986537d006320581f => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:46.235494 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf91ea-865f-400c-8d37-58b8e616b411 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:46.350052 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcf91ea-865f-400c-8d37-58b8e616b411 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:46.379473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:46.385976 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d7f9f-b7f0-41bb-a6b4-40ee37ed4766 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:46.386861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d7f9f-b7f0-41bb-a6b4-40ee37ed4766 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:46.387145 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d7f9f-b7f0-41bb-a6b4-40ee37ed4766 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:46.387418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db3d7f9f-b7f0-41bb-a6b4-40ee37ed4766 None admin] PATH_INFO: `/v3/users/2fd321cb4fce421a938aac2fe462e71a` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:46.388658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d7f9f-b7f0-41bb-a6b4-40ee37ed4766 None admin] RBAC: Authorizing `identity:get_user(user_id=2fd321cb4fce421a938aac2fe462e71a)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:46.390114 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db3d7f9f-b7f0-41bb-a6b4-40ee37ed4766 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:46.391730 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 558/1127] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:55:46 2022] GET /identity/v3/users/2fd321cb4fce421a938aac2fe462e71a => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:46.398259 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef2377-a30f-4ee0-9d29-7a066411d199 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:46.507610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:46.510926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ef2377-a30f-4ee0-9d29-7a066411d199 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:46.546183 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea95e4-c541-4d03-b297-91951ce27c88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:46.546830 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea95e4-c541-4d03-b297-91951ce27c88 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:46.547126 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea95e4-c541-4d03-b297-91951ce27c88 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:46.547412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aea95e4-c541-4d03-b297-91951ce27c88 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:46.621721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea95e4-c541-4d03-b297-91951ce27c88 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:46.623206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aea95e4-c541-4d03-b297-91951ce27c88 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:46.726120 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:46.732282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 570/1128] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:55:46 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:55:46.738361 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d7f9f-b7f0-41bb-a6b4-40ee37ed4766 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:46.842550 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db3d7f9f-b7f0-41bb-a6b4-40ee37ed4766 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:46.879699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653bfc59-2943-4fc9-ac5c-7d6619355a91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:46.880576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bfc59-2943-4fc9-ac5c-7d6619355a91 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:46.880857 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bfc59-2943-4fc9-ac5c-7d6619355a91 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:46.881134 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653bfc59-2943-4fc9-ac5c-7d6619355a91 None admin] PATH_INFO: `/v3/projects/c0881e3bcd064aa58e5a8187f5032cbd` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:46.882341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bfc59-2943-4fc9-ac5c-7d6619355a91 None admin] RBAC: Authorizing `identity:get_project(project_id=c0881e3bcd064aa58e5a8187f5032cbd)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:46.883850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653bfc59-2943-4fc9-ac5c-7d6619355a91 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:46.885525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 559/1129] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:55:46 2022] GET /identity/v3/projects/c0881e3bcd064aa58e5a8187f5032cbd => generated 311 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:46.894971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea95e4-c541-4d03-b297-91951ce27c88 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:47.007351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aea95e4-c541-4d03-b297-91951ce27c88 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:47.042217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84972cb9-442f-4a64-9a99-a75b23b053dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:47.042860 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84972cb9-442f-4a64-9a99-a75b23b053dc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:47.043158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84972cb9-442f-4a64-9a99-a75b23b053dc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:47.043436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84972cb9-442f-4a64-9a99-a75b23b053dc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:47.044289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84972cb9-442f-4a64-9a99-a75b23b053dc None admin] RBAC: Adding query filter params (role.id=1cc9b5d798c648c986537d006320581f, scope.project.id=c0881e3bcd064aa58e5a8187f5032cbd, user.id=2fd321cb4fce421a938aac2fe462e71a) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:47.044608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84972cb9-442f-4a64-9a99-a75b23b053dc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:47.046058 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84972cb9-442f-4a64-9a99-a75b23b053dc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:47.051992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 571/1130] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:55:46 2022] GET /identity/v3/role_assignments?user.id=2fd321cb4fce421a938aac2fe462e71a&scope.project.id=c0881e3bcd064aa58e5a8187f5032cbd&role.id=1cc9b5d798c648c986537d006320581f => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:49.235702 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad625d-e5bf-478e-a0cd-eb76e2d94d0e None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:49.236301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad625d-e5bf-478e-a0cd-eb76e2d94d0e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:49.236301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99ad625d-e5bf-478e-a0cd-eb76e2d94d0e None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:49.237672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 560/1131] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:49.244107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9bf9e-7c2d-4bc3-b889-688d3c117102 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:49.244830 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9bf9e-7c2d-4bc3-b889-688d3c117102 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:49.245009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d9bf9e-7c2d-4bc3-b889-688d3c117102 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:49.269039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c2d9bf9e-7c2d-4bc3-b889-688d3c117102 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:49.270240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c2d9bf9e-7c2d-4bc3-b889-688d3c117102 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:49.339655 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:49.349243 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 572/1132] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:49.372764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad625d-e5bf-478e-a0cd-eb76e2d94d0e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:49.445321 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:49.485550 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99ad625d-e5bf-478e-a0cd-eb76e2d94d0e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:49.520104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-894827cb-74d8-460d-8fd7-635d8f6ef22d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:49.520839 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894827cb-74d8-460d-8fd7-635d8f6ef22d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:49.521138 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894827cb-74d8-460d-8fd7-635d8f6ef22d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:49.521417 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-894827cb-74d8-460d-8fd7-635d8f6ef22d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:49.522628 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894827cb-74d8-460d-8fd7-635d8f6ef22d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:49.524193 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-894827cb-74d8-460d-8fd7-635d8f6ef22d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:49.527159 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 561/1133] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:49.533091 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9bf9e-7c2d-4bc3-b889-688d3c117102 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:49.644113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d9bf9e-7c2d-4bc3-b889-688d3c117102 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:49.679768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c68bf377-9cfe-4477-b794-359c25d96c61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:49.680514 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bf377-9cfe-4477-b794-359c25d96c61 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:49.680806 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bf377-9cfe-4477-b794-359c25d96c61 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:49.681087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c68bf377-9cfe-4477-b794-359c25d96c61 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:49.681799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bf377-9cfe-4477-b794-359c25d96c61 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:49.683281 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c68bf377-9cfe-4477-b794-359c25d96c61 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:49.714374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 573/1134] 10.222.0.85 () {60 vars in 1141 bytes} [Wed May 25 15:55:49 2022] POST /identity/v3/projects => generated 311 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:55:51.924226 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed4db2-e00e-4605-a3ff-8aaa219434fc None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:51.925086 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed4db2-e00e-4605-a3ff-8aaa219434fc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:51.925086 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ed4db2-e00e-4605-a3ff-8aaa219434fc None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:51.927325 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 562/1135] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:51.933165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a2f8f-6877-4b08-a2d9-4c4f58c85b25 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:51.933587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a2f8f-6877-4b08-a2d9-4c4f58c85b25 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:51.933756 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622a2f8f-6877-4b08-a2d9-4c4f58c85b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:51.952406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-622a2f8f-6877-4b08-a2d9-4c4f58c85b25 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:51.953942 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-622a2f8f-6877-4b08-a2d9-4c4f58c85b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:52.000062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:52.027094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 574/1136] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:51 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:52.052113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed4db2-e00e-4605-a3ff-8aaa219434fc None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:52.171012 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ed4db2-e00e-4605-a3ff-8aaa219434fc None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:52.206492 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a86037-5312-4a03-99d2-9021dc316feb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:52.207262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a86037-5312-4a03-99d2-9021dc316feb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:52.207545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a86037-5312-4a03-99d2-9021dc316feb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:52.207847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a86037-5312-4a03-99d2-9021dc316feb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:52.209080 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a86037-5312-4a03-99d2-9021dc316feb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:52.210529 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a86037-5312-4a03-99d2-9021dc316feb None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:52.212231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 563/1137] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:55:52 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:52.218207 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a2f8f-6877-4b08-a2d9-4c4f58c85b25 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:52.342957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622a2f8f-6877-4b08-a2d9-4c4f58c85b25 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:52.378962 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a721727-8ff1-48e3-b93e-342fb7e6b61d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:52.379650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a721727-8ff1-48e3-b93e-342fb7e6b61d None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:52.379970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a721727-8ff1-48e3-b93e-342fb7e6b61d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:52.380256 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a721727-8ff1-48e3-b93e-342fb7e6b61d None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:52.380974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a721727-8ff1-48e3-b93e-342fb7e6b61d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:52.382436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a721727-8ff1-48e3-b93e-342fb7e6b61d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:52.407056 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 575/1138] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:55:52 2022] POST /identity/v3/users => generated 286 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:55:54.612134 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcb577-7fb6-40fa-b30e-deecfbec1dc2 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:54.612660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcb577-7fb6-40fa-b30e-deecfbec1dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:54.612660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fcb577-7fb6-40fa-b30e-deecfbec1dc2 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:54.614710 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 564/1139] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:54.623062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b68c5-858e-42f9-982e-548746af14d1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:54.623505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b68c5-858e-42f9-982e-548746af14d1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:54.623616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156b68c5-858e-42f9-982e-548746af14d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:54.642036 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-156b68c5-858e-42f9-982e-548746af14d1 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:54.643233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-156b68c5-858e-42f9-982e-548746af14d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:54.670036 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:54.716515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 576/1140] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:54.744207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcb577-7fb6-40fa-b30e-deecfbec1dc2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:54.747370 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:54.855051 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fcb577-7fb6-40fa-b30e-deecfbec1dc2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:54.889804 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df96b1-3dcc-4d9b-9b8a-1e6bef194714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:54.890603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df96b1-3dcc-4d9b-9b8a-1e6bef194714 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:54.890892 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df96b1-3dcc-4d9b-9b8a-1e6bef194714 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:54.891172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df96b1-3dcc-4d9b-9b8a-1e6bef194714 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:54.901796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df96b1-3dcc-4d9b-9b8a-1e6bef194714 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:55:54.909503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-38df96b1-3dcc-4d9b-9b8a-1e6bef194714 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:55:54.909503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:54.909503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:54.909503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:54.909503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:54.909503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:54.909503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:54.909503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:54.909503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:54.909503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:54.909503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:54.909503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:54.909503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:55:54.909503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:55:54.909503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:55:54.909503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:55:54.910629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:55:54.910629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:55:54.910629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:55:54.910629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:55:54.910629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:54.910629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:54.910629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:54.910629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:54.910629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:54.910629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:54.910629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:54.910629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:54.910629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:54.910629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:54.910629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:54.910629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:54.911663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:54.911663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:54.911663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:54.911663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:54.911663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:55:54.911663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:55:54.911663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:55:54.911663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:55:54.911663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:55:54.911663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:55:54.911663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:55:54.911663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:54.911663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 565/1141] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:55:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:54.917426 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b68c5-858e-42f9-982e-548746af14d1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:55.026200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156b68c5-858e-42f9-982e-548746af14d1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:55.065567 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a1212-c6f3-45c5-b86f-02fe9445e9f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:55.066235 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a1212-c6f3-45c5-b86f-02fe9445e9f6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:55.066523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a1212-c6f3-45c5-b86f-02fe9445e9f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:55.066807 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520a1212-c6f3-45c5-b86f-02fe9445e9f6 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:55.067574 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a1212-c6f3-45c5-b86f-02fe9445e9f6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:55.067925 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a1212-c6f3-45c5-b86f-02fe9445e9f6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:55.069394 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520a1212-c6f3-45c5-b86f-02fe9445e9f6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:55.078017 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 577/1142] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:55:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:55.084221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df96b1-3dcc-4d9b-9b8a-1e6bef194714 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:55.149359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:55.192691 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df96b1-3dcc-4d9b-9b8a-1e6bef194714 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:55.227507 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b517c0-ac5a-4165-8cf2-047ff482ebf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:55.228419 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b517c0-ac5a-4165-8cf2-047ff482ebf9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:55.228711 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b517c0-ac5a-4165-8cf2-047ff482ebf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:55.229004 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b517c0-ac5a-4165-8cf2-047ff482ebf9 None admin] PATH_INFO: `/v3/users/3478654e7c484f5b873f044b7ed0fc97` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:55.242423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b517c0-ac5a-4165-8cf2-047ff482ebf9 None admin] RBAC: Authorizing `identity:get_user(user_id=3478654e7c484f5b873f044b7ed0fc97)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:55.243863 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b517c0-ac5a-4165-8cf2-047ff482ebf9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:55.245560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 566/1143] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:55:55 2022] GET /identity/v3/users/3478654e7c484f5b873f044b7ed0fc97 => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:55.251598 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a1212-c6f3-45c5-b86f-02fe9445e9f6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:55.281293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:55.359640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520a1212-c6f3-45c5-b86f-02fe9445e9f6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:55.400102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6f010-a731-4e73-ad12-62758df05f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:55.400677 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6f010-a731-4e73-ad12-62758df05f06 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:55.400978 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6f010-a731-4e73-ad12-62758df05f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:55.401253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da6f010-a731-4e73-ad12-62758df05f06 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:55.471690 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6f010-a731-4e73-ad12-62758df05f06 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:55.473200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da6f010-a731-4e73-ad12-62758df05f06 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:55.496851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:55.573773 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 578/1144] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:55:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:55:55.580311 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b517c0-ac5a-4165-8cf2-047ff482ebf9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:55.691402 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b517c0-ac5a-4165-8cf2-047ff482ebf9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:55.712095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:55.727232 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea98dae-ff30-4220-9c94-9cdb11765bc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:55.728144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea98dae-ff30-4220-9c94-9cdb11765bc2 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:55.728437 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea98dae-ff30-4220-9c94-9cdb11765bc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:55.728714 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea98dae-ff30-4220-9c94-9cdb11765bc2 None admin] PATH_INFO: `/v3/projects/9a79eb30848c4c15a640736bb3359bdd` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:55.729988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea98dae-ff30-4220-9c94-9cdb11765bc2 None admin] RBAC: Authorizing `identity:get_project(project_id=9a79eb30848c4c15a640736bb3359bdd)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:55.731441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea98dae-ff30-4220-9c94-9cdb11765bc2 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:55.733194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 567/1145] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:55:55 2022] GET /identity/v3/projects/9a79eb30848c4c15a640736bb3359bdd => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:55.739861 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6f010-a731-4e73-ad12-62758df05f06 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:55.851030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da6f010-a731-4e73-ad12-62758df05f06 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:55.875524 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:55.887717 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de48ffa1-7c8a-4130-8d1e-8f5643d88613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:55.888408 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48ffa1-7c8a-4130-8d1e-8f5643d88613 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:55.888701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48ffa1-7c8a-4130-8d1e-8f5643d88613 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:55.888986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de48ffa1-7c8a-4130-8d1e-8f5643d88613 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:55.889826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48ffa1-7c8a-4130-8d1e-8f5643d88613 None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=9a79eb30848c4c15a640736bb3359bdd, user.id=3478654e7c484f5b873f044b7ed0fc97) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:55.890185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48ffa1-7c8a-4130-8d1e-8f5643d88613 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:55.891646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de48ffa1-7c8a-4130-8d1e-8f5643d88613 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:55.897431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 579/1146] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:55:55 2022] GET /identity/v3/role_assignments?user.id=3478654e7c484f5b873f044b7ed0fc97&scope.project.id=9a79eb30848c4c15a640736bb3359bdd&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:58.027995 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cab343-1634-4867-924c-3694f61289c3 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:58.028580 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cab343-1634-4867-924c-3694f61289c3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:58.028580 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cab343-1634-4867-924c-3694f61289c3 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:58.029938 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 568/1147] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:55:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:55:58.036675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a062a-ba41-4527-86db-441542ed01b5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:58.037435 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a062a-ba41-4527-86db-441542ed01b5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:58.037666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6a062a-ba41-4527-86db-441542ed01b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:58.062231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-df6a062a-ba41-4527-86db-441542ed01b5 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:55:58.063422 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-df6a062a-ba41-4527-86db-441542ed01b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:55:58.135222 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 580/1148] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:55:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:55:58.168790 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cab343-1634-4867-924c-3694f61289c3 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:58.273995 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cab343-1634-4867-924c-3694f61289c3 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:58.309054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3d882-b392-4233-80b8-ba8725a8c91c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:58.309851 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3d882-b392-4233-80b8-ba8725a8c91c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:58.310141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3d882-b392-4233-80b8-ba8725a8c91c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:58.310412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7df3d882-b392-4233-80b8-ba8725a8c91c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:58.321978 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7df3d882-b392-4233-80b8-ba8725a8c91c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:55:58.329923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-7df3d882-b392-4233-80b8-ba8725a8c91c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:55:58.329923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:55:58.329923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:55:58.329923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:55:58.329923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:55:58.329923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:55:58.329923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:55:58.329923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:55:58.329923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:55:58.329923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:55:58.329923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:55:58.329923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:55:58.329923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:55:58.329923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:55:58.329923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:55:58.329923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:55:58.331026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:55:58.331026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:55:58.331026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:55:58.331026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:55:58.331026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:55:58.331026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:55:58.331026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:55:58.331026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:55:58.331026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:55:58.331026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:55:58.331026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:55:58.331026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:55:58.331026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:55:58.331026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:55:58.331026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:55:58.331026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:55:58.332055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:55:58.332055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:55:58.332055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:55:58.332055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:55:58.332055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:55:58.332055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:55:58.332055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:55:58.332055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:55:58.332055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:55:58.332055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:55:58.332055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:55:58.332055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:55:58.336033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 569/1149] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:55:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:55:58.337238 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a062a-ba41-4527-86db-441542ed01b5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:58.439407 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6a062a-ba41-4527-86db-441542ed01b5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:58.457099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:58.474008 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6ce79-2304-4ae2-b236-a0a2281567dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:58.474691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6ce79-2304-4ae2-b236-a0a2281567dc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:58.474981 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6ce79-2304-4ae2-b236-a0a2281567dc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:58.475266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c6ce79-2304-4ae2-b236-a0a2281567dc None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:58.476065 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6ce79-2304-4ae2-b236-a0a2281567dc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:55:58.476436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6ce79-2304-4ae2-b236-a0a2281567dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:58.477844 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c6ce79-2304-4ae2-b236-a0a2281567dc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:58.485853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 581/1150] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:55:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:58.492031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3d882-b392-4233-80b8-ba8725a8c91c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:58.544454 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:58.595034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7df3d882-b392-4233-80b8-ba8725a8c91c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:58.628678 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efcacf-b600-46e2-a703-3a5b82a10014 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:58.629573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efcacf-b600-46e2-a703-3a5b82a10014 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:58.629894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efcacf-b600-46e2-a703-3a5b82a10014 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:58.630179 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62efcacf-b600-46e2-a703-3a5b82a10014 None admin] PATH_INFO: `/v3/users/3478654e7c484f5b873f044b7ed0fc97` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:58.631428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62efcacf-b600-46e2-a703-3a5b82a10014 None admin] RBAC: Authorizing `identity:get_user(user_id=3478654e7c484f5b873f044b7ed0fc97)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:58.633073 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62efcacf-b600-46e2-a703-3a5b82a10014 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:58.634809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 570/1151] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:55:58 2022] GET /identity/v3/users/3478654e7c484f5b873f044b7ed0fc97 => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:58.643198 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6ce79-2304-4ae2-b236-a0a2281567dc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:58.748645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c6ce79-2304-4ae2-b236-a0a2281567dc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:58.789250 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ed5b67-4cd5-4760-96d9-d5b721b54bb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:58.790112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed5b67-4cd5-4760-96d9-d5b721b54bb5 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:58.790396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed5b67-4cd5-4760-96d9-d5b721b54bb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:58.790670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ed5b67-4cd5-4760-96d9-d5b721b54bb5 None admin] PATH_INFO: `/v3/projects/9a79eb30848c4c15a640736bb3359bdd` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:58.791991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed5b67-4cd5-4760-96d9-d5b721b54bb5 None admin] RBAC: Authorizing `identity:get_project(project_id=9a79eb30848c4c15a640736bb3359bdd)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:58.793461 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ed5b67-4cd5-4760-96d9-d5b721b54bb5 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:58.795125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 582/1152] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:55:58 2022] GET /identity/v3/projects/9a79eb30848c4c15a640736bb3359bdd => generated 311 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:55:58.804264 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efcacf-b600-46e2-a703-3a5b82a10014 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:55:58.905941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62efcacf-b600-46e2-a703-3a5b82a10014 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:55:58.921957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:55:58.942071 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c92076-f946-47c2-9d81-dea3e74aa7f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:55:58.942963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c92076-f946-47c2-9d81-dea3e74aa7f1 None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:55:58.943255 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c92076-f946-47c2-9d81-dea3e74aa7f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:55:58.943536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c92076-f946-47c2-9d81-dea3e74aa7f1 None admin] PATH_INFO: `/v3/projects/9a79eb30848c4c15a640736bb3359bdd/users/3478654e7c484f5b873f044b7ed0fc97/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:55:58.945823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c92076-f946-47c2-9d81-dea3e74aa7f1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9a79eb30848c4c15a640736bb3359bdd, user_id=3478654e7c484f5b873f044b7ed0fc97, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:55:58.947266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c92076-f946-47c2-9d81-dea3e74aa7f1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:55:58.963328 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 571/1153] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:55:58 2022] PUT /identity/v3/projects/9a79eb30848c4c15a640736bb3359bdd/users/3478654e7c484f5b873f044b7ed0fc97/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:56:01.154385 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50f5b3-a215-46d2-a936-65f4839163a4 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:01.155285 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50f5b3-a215-46d2-a936-65f4839163a4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:01.155285 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe50f5b3-a215-46d2-a936-65f4839163a4 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:01.157523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 583/1154] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:56:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:56:01.163938 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811876a-d592-4f54-8c74-4a4671e00658 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:01.164402 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811876a-d592-4f54-8c74-4a4671e00658 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:01.164518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2811876a-d592-4f54-8c74-4a4671e00658 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:01.183122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-2811876a-d592-4f54-8c74-4a4671e00658 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:56:01.184362 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2811876a-d592-4f54-8c74-4a4671e00658 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:56:01.263753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 572/1155] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:56:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:56:01.287363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50f5b3-a215-46d2-a936-65f4839163a4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:01.344075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:01.399065 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe50f5b3-a215-46d2-a936-65f4839163a4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:01.434352 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a93f43-4fb2-4299-9be9-99294240e147 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:01.435082 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a93f43-4fb2-4299-9be9-99294240e147 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:01.435369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a93f43-4fb2-4299-9be9-99294240e147 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:01.435646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a93f43-4fb2-4299-9be9-99294240e147 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:01.445827 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a93f43-4fb2-4299-9be9-99294240e147 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:56:01.454187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-12a93f43-4fb2-4299-9be9-99294240e147 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:56:01.454187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:56:01.454187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:56:01.454187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:56:01.454187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:56:01.454187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:56:01.454187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:56:01.454187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:56:01.454187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:56:01.454187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:56:01.454187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:56:01.454187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:56:01.454187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:56:01.454187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:56:01.454187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:56:01.454187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:56:01.455306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:56:01.455306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:56:01.455306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:56:01.455306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:56:01.455306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:56:01.455306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:56:01.455306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:56:01.455306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:56:01.455306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:56:01.455306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:56:01.455306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:56:01.455306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:56:01.455306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:56:01.455306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:56:01.455306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:56:01.455306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:56:01.456398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:56:01.456398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:56:01.456398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:56:01.456398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:56:01.456398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:56:01.456398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:56:01.456398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:56:01.456398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:56:01.456398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:56:01.456398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:56:01.456398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:56:01.456398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:56:01.460006 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 584/1156] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:56:01 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:56:01.462298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811876a-d592-4f54-8c74-4a4671e00658 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:01.568921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2811876a-d592-4f54-8c74-4a4671e00658 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:01.579193 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:01.609679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0de51-fa59-47ac-8542-d144cf837fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:01.610352 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0de51-fa59-47ac-8542-d144cf837fde None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:01.610640 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0de51-fa59-47ac-8542-d144cf837fde None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:01.610922 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c0de51-fa59-47ac-8542-d144cf837fde None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:01.611671 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0de51-fa59-47ac-8542-d144cf837fde None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:56:01.612016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0de51-fa59-47ac-8542-d144cf837fde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:01.613396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c0de51-fa59-47ac-8542-d144cf837fde None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:01.622666 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 573/1157] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:56:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:01.629923 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a93f43-4fb2-4299-9be9-99294240e147 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:01.742032 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a93f43-4fb2-4299-9be9-99294240e147 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:01.750871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:01.777446 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd1125-0bdb-4eb8-ae7f-b823e1455f3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:01.778324 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd1125-0bdb-4eb8-ae7f-b823e1455f3d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:01.778611 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd1125-0bdb-4eb8-ae7f-b823e1455f3d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:01.778893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22dd1125-0bdb-4eb8-ae7f-b823e1455f3d None admin] PATH_INFO: `/v3/users/3478654e7c484f5b873f044b7ed0fc97` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:01.780168 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd1125-0bdb-4eb8-ae7f-b823e1455f3d None admin] RBAC: Authorizing `identity:get_user(user_id=3478654e7c484f5b873f044b7ed0fc97)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:01.781632 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22dd1125-0bdb-4eb8-ae7f-b823e1455f3d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:01.783256 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 585/1158] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:56:01 2022] GET /identity/v3/users/3478654e7c484f5b873f044b7ed0fc97 => generated 286 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:01.789978 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0de51-fa59-47ac-8542-d144cf837fde None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:01.901358 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c0de51-fa59-47ac-8542-d144cf837fde None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:01.936049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4d6ce-ecaf-4ec6-bd88-ce8b0e9a3c1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:01.936696 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4d6ce-ecaf-4ec6-bd88-ce8b0e9a3c1c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:01.936980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4d6ce-ecaf-4ec6-bd88-ce8b0e9a3c1c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:01.937258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c4d6ce-ecaf-4ec6-bd88-ce8b0e9a3c1c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:01.993769 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:02.031957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4d6ce-ecaf-4ec6-bd88-ce8b0e9a3c1c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:02.033434 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c4d6ce-ecaf-4ec6-bd88-ce8b0e9a3c1c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:02.139609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 574/1159] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:56:01 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:56:02.149183 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd1125-0bdb-4eb8-ae7f-b823e1455f3d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:02.265284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22dd1125-0bdb-4eb8-ae7f-b823e1455f3d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:02.302613 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97faf186-7067-4370-a0f6-70054aadd490 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:02.304445 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faf186-7067-4370-a0f6-70054aadd490 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:02.304445 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faf186-7067-4370-a0f6-70054aadd490 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:02.304445 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97faf186-7067-4370-a0f6-70054aadd490 None admin] PATH_INFO: `/v3/projects/9a79eb30848c4c15a640736bb3359bdd` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:02.305703 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97faf186-7067-4370-a0f6-70054aadd490 None admin] RBAC: Authorizing `identity:get_project(project_id=9a79eb30848c4c15a640736bb3359bdd)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:02.307177 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97faf186-7067-4370-a0f6-70054aadd490 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:02.308935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 586/1160] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:56:02 2022] GET /identity/v3/projects/9a79eb30848c4c15a640736bb3359bdd => generated 311 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:02.318387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4d6ce-ecaf-4ec6-bd88-ce8b0e9a3c1c None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:02.398391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:02.446029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c4d6ce-ecaf-4ec6-bd88-ce8b0e9a3c1c None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:02.482205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7383f587-ac57-4143-abb5-aa122e0011de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:02.482863 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383f587-ac57-4143-abb5-aa122e0011de None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:02.483161 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383f587-ac57-4143-abb5-aa122e0011de None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:02.483441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7383f587-ac57-4143-abb5-aa122e0011de None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:02.484319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383f587-ac57-4143-abb5-aa122e0011de None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=9a79eb30848c4c15a640736bb3359bdd, user.id=3478654e7c484f5b873f044b7ed0fc97) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:56:02.484640 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383f587-ac57-4143-abb5-aa122e0011de None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:02.486088 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7383f587-ac57-4143-abb5-aa122e0011de None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:02.492327 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 575/1161] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:56:02 2022] GET /identity/v3/role_assignments?user.id=3478654e7c484f5b873f044b7ed0fc97&scope.project.id=9a79eb30848c4c15a640736bb3359bdd&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 608 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:04.682388 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a002b7-d944-437f-a547-ead2a2f79506 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:04.682388 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a002b7-d944-437f-a547-ead2a2f79506 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:04.682388 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69a002b7-d944-437f-a547-ead2a2f79506 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:04.683207 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 587/1162] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:56:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:56:04.689345 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b679853-83c4-4c40-9c85-ac2935b1df75 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:04.690054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b679853-83c4-4c40-9c85-ac2935b1df75 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:04.690241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b679853-83c4-4c40-9c85-ac2935b1df75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:04.715549 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-5b679853-83c4-4c40-9c85-ac2935b1df75 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:56:04.716788 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5b679853-83c4-4c40-9c85-ac2935b1df75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:56:04.785760 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 576/1163] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:56:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:56:04.808433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a002b7-d944-437f-a547-ead2a2f79506 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:04.861415 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:04.925894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69a002b7-d944-437f-a547-ead2a2f79506 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:04.978564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80819870-a615-460c-bab7-7eb296971de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:04.979156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80819870-a615-460c-bab7-7eb296971de9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:04.979447 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80819870-a615-460c-bab7-7eb296971de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:04.979730 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80819870-a615-460c-bab7-7eb296971de9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:05.052562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80819870-a615-460c-bab7-7eb296971de9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:05.054068 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80819870-a615-460c-bab7-7eb296971de9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:05.099077 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:05.166395 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 588/1164] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:56:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:56:05.176346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b679853-83c4-4c40-9c85-ac2935b1df75 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:05.247724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:05.288918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b679853-83c4-4c40-9c85-ac2935b1df75 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:05.329987 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818dd476-bdaa-41fa-8c38-b5e1e8197a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:05.330807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dd476-bdaa-41fa-8c38-b5e1e8197a6b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:05.331100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dd476-bdaa-41fa-8c38-b5e1e8197a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:05.331406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818dd476-bdaa-41fa-8c38-b5e1e8197a6b None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:05.338519 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818dd476-bdaa-41fa-8c38-b5e1e8197a6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:05.340081 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818dd476-bdaa-41fa-8c38-b5e1e8197a6b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:05.357625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-818dd476-bdaa-41fa-8c38-b5e1e8197a6b None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. May 25 15:56:05.357625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:56:05.357625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain May 25 15:56:05.357625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) May 25 15:56:05.357625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:56:05.357625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:56:05.357625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:56:05.357625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:56:05.357625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. May 25 15:56:05.357625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:56:05.357625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:56:05.357625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:56:05.357625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:56:05.357625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:56:05.357625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:56:05.357625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:56:05.357625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:56:05.358912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:56:05.358912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:56:05.358912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:56:05.358912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:56:05.358912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:56:05.358912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:56:05.358912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:56:05.358912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:56:05.358912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:56:05.358912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:56:05.358912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:56:05.358912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:56:05.358912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:56:05.358912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:56:05.358912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:56:05.358912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:56:05.359860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:56:05.359860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:56:05.359860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:56:05.359860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:56:05.359860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:56:05.359860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:56:05.359860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:56:05.359860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:56:05.359860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:56:05.359860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:56:05.359860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain May 25 15:56:05.359860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:56:05.359860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. May 25 15:56:05.359860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:56:05.359860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 577/1165] 10.222.0.85 () {56 vars in 1117 bytes} [Wed May 25 15:56:05 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:56:05.369427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80819870-a615-460c-bab7-7eb296971de9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:05.480023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80819870-a615-460c-bab7-7eb296971de9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:05.516471 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed98c496-3bf5-49c2-9201-eed94263b0ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:05.517067 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98c496-3bf5-49c2-9201-eed94263b0ec None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:05.517355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98c496-3bf5-49c2-9201-eed94263b0ec None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:05.517637 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed98c496-3bf5-49c2-9201-eed94263b0ec None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:05.518371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98c496-3bf5-49c2-9201-eed94263b0ec None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:56:05.518681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98c496-3bf5-49c2-9201-eed94263b0ec None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:05.520093 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed98c496-3bf5-49c2-9201-eed94263b0ec None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:05.527630 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 589/1166] 10.222.0.85 () {56 vars in 1115 bytes} [Wed May 25 15:56:05 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:07.664268 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4339ee1-9f75-474d-8a7a-462f62d8f411 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:07.664858 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4339ee1-9f75-474d-8a7a-462f62d8f411 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:07.664858 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4339ee1-9f75-474d-8a7a-462f62d8f411 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:07.666191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 578/1167] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:56:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:56:07.673068 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ad097-3c92-4bf8-a32f-d171637878ad None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:07.673761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ad097-3c92-4bf8-a32f-d171637878ad None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:07.674010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c35ad097-3c92-4bf8-a32f-d171637878ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:07.693282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c35ad097-3c92-4bf8-a32f-d171637878ad None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:56:07.694473 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c35ad097-3c92-4bf8-a32f-d171637878ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:56:07.696543 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:07.772804 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 590/1168] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:56:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:56:07.797513 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4339ee1-9f75-474d-8a7a-462f62d8f411 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:07.915845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4339ee1-9f75-474d-8a7a-462f62d8f411 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:07.939811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:07.956708 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a535746-c2c3-4afc-85b7-e71ffa73ce66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:07.957676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a535746-c2c3-4afc-85b7-e71ffa73ce66 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:07.958058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a535746-c2c3-4afc-85b7-e71ffa73ce66 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:07.958406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a535746-c2c3-4afc-85b7-e71ffa73ce66 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:07.959512 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a535746-c2c3-4afc-85b7-e71ffa73ce66 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:07.961749 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a535746-c2c3-4afc-85b7-e71ffa73ce66 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:07.980950 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 579/1169] 10.222.0.85 () {60 vars in 1138 bytes} [Wed May 25 15:56:07 2022] POST /identity/v3/domains => generated 267 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:56:10.190040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0b804-ed3d-46c8-a748-ac2a32ec1892 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:10.190628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0b804-ed3d-46c8-a748-ac2a32ec1892 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:10.190628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0a0b804-ed3d-46c8-a748-ac2a32ec1892 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:10.192757 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 591/1170] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:56:10 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:56:10.202409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812267a-45f5-4fdd-98ba-3c24e446bb67 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:10.202827 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812267a-45f5-4fdd-98ba-3c24e446bb67 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:10.203049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812267a-45f5-4fdd-98ba-3c24e446bb67 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:10.228657 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b812267a-45f5-4fdd-98ba-3c24e446bb67 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:56:10.229850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b812267a-45f5-4fdd-98ba-3c24e446bb67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:56:10.298727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 580/1171] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:56:10 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:56:10.321900 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0b804-ed3d-46c8-a748-ac2a32ec1892 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:10.430697 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0a0b804-ed3d-46c8-a748-ac2a32ec1892 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:10.467338 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ee7dcb-0968-4b9e-a418-8f76b595c843 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:10.468178 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee7dcb-0968-4b9e-a418-8f76b595c843 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:10.468473 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee7dcb-0968-4b9e-a418-8f76b595c843 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:10.468753 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ee7dcb-0968-4b9e-a418-8f76b595c843 None admin] PATH_INFO: `/v3/domains/43080856c0f44b73a94a1ec7321167d1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:10.475891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:10.480797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee7dcb-0968-4b9e-a418-8f76b595c843 None admin] RBAC: Authorizing `identity:get_domain(domain_id=43080856c0f44b73a94a1ec7321167d1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:10.482331 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ee7dcb-0968-4b9e-a418-8f76b595c843 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:10.484012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 592/1172] 10.222.0.85 () {56 vars in 1183 bytes} [Wed May 25 15:56:10 2022] GET /identity/v3/domains/43080856c0f44b73a94a1ec7321167d1 => generated 267 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:10.490175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812267a-45f5-4fdd-98ba-3c24e446bb67 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:10.592460 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812267a-45f5-4fdd-98ba-3c24e446bb67 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:10.605497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:10.628948 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3746f64c-5903-4b7b-b808-6b05a81feb95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:10.629600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746f64c-5903-4b7b-b808-6b05a81feb95 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:10.629888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746f64c-5903-4b7b-b808-6b05a81feb95 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:10.630170 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3746f64c-5903-4b7b-b808-6b05a81feb95 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:10.630869 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746f64c-5903-4b7b-b808-6b05a81feb95 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:10.632371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3746f64c-5903-4b7b-b808-6b05a81feb95 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:10.660599 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 581/1173] 10.222.0.85 () {60 vars in 1141 bytes} [Wed May 25 15:56:10 2022] POST /identity/v3/projects => generated 361 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:56:12.900793 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd1a61-ee02-47f4-b381-54665dd55dd5 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:12.901390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd1a61-ee02-47f4-b381-54665dd55dd5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:12.901390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cd1a61-ee02-47f4-b381-54665dd55dd5 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:12.902752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 593/1174] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:56:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:56:12.909278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab754d-e58d-4600-a74c-c7533d5c8fa7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:12.910004 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab754d-e58d-4600-a74c-c7533d5c8fa7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:12.910134 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ab754d-e58d-4600-a74c-c7533d5c8fa7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:12.934260 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e5ab754d-e58d-4600-a74c-c7533d5c8fa7 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:56:12.935445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e5ab754d-e58d-4600-a74c-c7533d5c8fa7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:56:13.008211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 582/1175] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:56:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:56:13.038766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd1a61-ee02-47f4-b381-54665dd55dd5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:13.148573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cd1a61-ee02-47f4-b381-54665dd55dd5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:13.187107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f87887-043a-4b31-9bca-d5089f298592 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:13.187935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f87887-043a-4b31-9bca-d5089f298592 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:13.188224 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f87887-043a-4b31-9bca-d5089f298592 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:13.188506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f87887-043a-4b31-9bca-d5089f298592 None admin] PATH_INFO: `/v3/domains/43080856c0f44b73a94a1ec7321167d1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:13.189823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f87887-043a-4b31-9bca-d5089f298592 None admin] RBAC: Authorizing `identity:get_domain(domain_id=43080856c0f44b73a94a1ec7321167d1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:13.191283 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6f87887-043a-4b31-9bca-d5089f298592 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:13.192924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 594/1176] 10.222.0.85 () {56 vars in 1183 bytes} [Wed May 25 15:56:13 2022] GET /identity/v3/domains/43080856c0f44b73a94a1ec7321167d1 => generated 267 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:13.199637 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab754d-e58d-4600-a74c-c7533d5c8fa7 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:13.304758 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:13.313380 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ab754d-e58d-4600-a74c-c7533d5c8fa7 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:13.348345 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f984883-9adb-4b67-9f89-97967521131a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:13.349014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f984883-9adb-4b67-9f89-97967521131a None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:13.349295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f984883-9adb-4b67-9f89-97967521131a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:13.349573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f984883-9adb-4b67-9f89-97967521131a None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:13.350281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f984883-9adb-4b67-9f89-97967521131a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:13.351721 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f984883-9adb-4b67-9f89-97967521131a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:13.379711 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 583/1177] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:56:13 2022] POST /identity/v3/users => generated 311 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:56:15.595970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485933b7-82a8-4c43-bca9-7dfe817b3f6f None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:15.596498 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485933b7-82a8-4c43-bca9-7dfe817b3f6f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:15.596498 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485933b7-82a8-4c43-bca9-7dfe817b3f6f None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:15.597860 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 595/1178] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:56:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:56:15.604719 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0b57f-a813-4373-ac11-d7f8ab25bf42 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:15.605157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0b57f-a813-4373-ac11-d7f8ab25bf42 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:15.605308 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc0b57f-a813-4373-ac11-d7f8ab25bf42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:15.631406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4fc0b57f-a813-4373-ac11-d7f8ab25bf42 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:56:15.632629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4fc0b57f-a813-4373-ac11-d7f8ab25bf42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:56:15.699742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 584/1179] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:56:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:56:15.730967 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485933b7-82a8-4c43-bca9-7dfe817b3f6f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:15.812130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:15.857593 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485933b7-82a8-4c43-bca9-7dfe817b3f6f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:15.904339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e5791-abcc-4e87-8be1-aba22e645e41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:15.905146 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e5791-abcc-4e87-8be1-aba22e645e41 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:15.905438 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e5791-abcc-4e87-8be1-aba22e645e41 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:15.905717 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7e5791-abcc-4e87-8be1-aba22e645e41 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:15.916838 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7e5791-abcc-4e87-8be1-aba22e645e41 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:56:15.924563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3a7e5791-abcc-4e87-8be1-aba22e645e41 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:56:15.924563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:56:15.924563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:56:15.924563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:56:15.924563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:56:15.924563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:56:15.924563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:56:15.924563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:56:15.924563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:56:15.924563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:56:15.924563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:56:15.924563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:56:15.924563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:56:15.924563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:56:15.924563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:56:15.924563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:56:15.925627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:56:15.925627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:56:15.925627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:56:15.925627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:56:15.925627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:56:15.925627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:56:15.925627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:56:15.925627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:56:15.925627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:56:15.925627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:56:15.925627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:56:15.925627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:56:15.925627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:56:15.925627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:56:15.925627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:56:15.925627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:56:15.926649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:56:15.926649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:56:15.926649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:56:15.926649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:56:15.926649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:56:15.926649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:56:15.926649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:56:15.926649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:56:15.926649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:56:15.926649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:56:15.926649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:56:15.926649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:56:15.926649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 596/1180] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:56:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:56:15.932106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0b57f-a813-4373-ac11-d7f8ab25bf42 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:16.029358 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:16.058283 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc0b57f-a813-4373-ac11-d7f8ab25bf42 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:16.096058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4906e5-b942-44ec-aaba-b90543771393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:16.096735 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4906e5-b942-44ec-aaba-b90543771393 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:16.097021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4906e5-b942-44ec-aaba-b90543771393 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:16.097305 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4906e5-b942-44ec-aaba-b90543771393 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:16.098052 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4906e5-b942-44ec-aaba-b90543771393 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:56:16.098368 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4906e5-b942-44ec-aaba-b90543771393 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:16.099755 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4906e5-b942-44ec-aaba-b90543771393 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:16.107807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 585/1181] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:56:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:16.117155 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e5791-abcc-4e87-8be1-aba22e645e41 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:16.240467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7e5791-abcc-4e87-8be1-aba22e645e41 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:16.269460 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:16.280067 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890308a-8e36-4bc1-a7de-745ef6d614b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:16.280541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890308a-8e36-4bc1-a7de-745ef6d614b0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:16.280841 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890308a-8e36-4bc1-a7de-745ef6d614b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:16.281120 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4890308a-8e36-4bc1-a7de-745ef6d614b0 None admin] PATH_INFO: `/v3/users/57efc43eb6e5468080c26a8cc6f714f8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:16.297681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4890308a-8e36-4bc1-a7de-745ef6d614b0 None admin] RBAC: Authorizing `identity:get_user(user_id=57efc43eb6e5468080c26a8cc6f714f8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:16.299453 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4890308a-8e36-4bc1-a7de-745ef6d614b0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:16.301210 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 597/1182] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:56:16 2022] GET /identity/v3/users/57efc43eb6e5468080c26a8cc6f714f8 => generated 311 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:16.308941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4906e5-b942-44ec-aaba-b90543771393 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:16.422515 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4906e5-b942-44ec-aaba-b90543771393 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:16.457541 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00b50b-6338-4b88-8f82-25399134c2cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:16.458181 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00b50b-6338-4b88-8f82-25399134c2cf None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:16.458469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00b50b-6338-4b88-8f82-25399134c2cf None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:16.458744 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f00b50b-6338-4b88-8f82-25399134c2cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:16.470647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:16.541294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00b50b-6338-4b88-8f82-25399134c2cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:16.542755 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f00b50b-6338-4b88-8f82-25399134c2cf None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:16.642382 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 586/1183] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:56:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:56:16.651639 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890308a-8e36-4bc1-a7de-745ef6d614b0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:16.756104 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4890308a-8e36-4bc1-a7de-745ef6d614b0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:16.791441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25b5c1b1-85d7-4a12-8d8a-38f5c2a28c0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:16.792338 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5c1b1-85d7-4a12-8d8a-38f5c2a28c0e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:16.792629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5c1b1-85d7-4a12-8d8a-38f5c2a28c0e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:16.792912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25b5c1b1-85d7-4a12-8d8a-38f5c2a28c0e None admin] PATH_INFO: `/v3/projects/31efd53599324dea97d78801918731f0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:16.794231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5c1b1-85d7-4a12-8d8a-38f5c2a28c0e None admin] RBAC: Authorizing `identity:get_project(project_id=31efd53599324dea97d78801918731f0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:16.795680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25b5c1b1-85d7-4a12-8d8a-38f5c2a28c0e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:16.797376 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 598/1184] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:56:16 2022] GET /identity/v3/projects/31efd53599324dea97d78801918731f0 => generated 361 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:16.803660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00b50b-6338-4b88-8f82-25399134c2cf None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:16.850851 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:16.908672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f00b50b-6338-4b88-8f82-25399134c2cf None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:16.944577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d31bd-7665-42d8-8974-54e1b390e567 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:16.945225 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d31bd-7665-42d8-8974-54e1b390e567 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:16.945520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d31bd-7665-42d8-8974-54e1b390e567 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:16.945851 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d31bd-7665-42d8-8974-54e1b390e567 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:16.946683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d31bd-7665-42d8-8974-54e1b390e567 None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=31efd53599324dea97d78801918731f0, user.id=57efc43eb6e5468080c26a8cc6f714f8) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:56:16.947005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d31bd-7665-42d8-8974-54e1b390e567 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:16.948486 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d31bd-7665-42d8-8974-54e1b390e567 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:16.954871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 587/1185] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:56:16 2022] GET /identity/v3/role_assignments?user.id=57efc43eb6e5468080c26a8cc6f714f8&scope.project.id=31efd53599324dea97d78801918731f0&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:19.109830 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff5e3-3ead-4c6c-b731-5be629f79aaa None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:19.110743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff5e3-3ead-4c6c-b731-5be629f79aaa None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:19.110743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638ff5e3-3ead-4c6c-b731-5be629f79aaa None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:19.112979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 599/1186] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:56:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:56:19.118467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b17746-2c5c-4d5a-904d-57aab161aa0c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:19.118898 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b17746-2c5c-4d5a-904d-57aab161aa0c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:19.119003 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b17746-2c5c-4d5a-904d-57aab161aa0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:19.137387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e5b17746-2c5c-4d5a-904d-57aab161aa0c None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:56:19.138586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e5b17746-2c5c-4d5a-904d-57aab161aa0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:56:19.208291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 588/1187] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:56:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:56:19.231509 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ff5e3-3ead-4c6c-b731-5be629f79aaa None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:19.296185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:19.359230 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638ff5e3-3ead-4c6c-b731-5be629f79aaa None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:19.403133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2693b-a7b2-4608-a71b-8901df9de989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:19.403963 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2693b-a7b2-4608-a71b-8901df9de989 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:19.404252 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2693b-a7b2-4608-a71b-8901df9de989 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:19.404526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f2693b-a7b2-4608-a71b-8901df9de989 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:19.415387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f2693b-a7b2-4608-a71b-8901df9de989 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:56:19.422992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-10f2693b-a7b2-4608-a71b-8901df9de989 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:56:19.422992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:56:19.422992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:56:19.422992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:56:19.422992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:56:19.422992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:56:19.422992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:56:19.422992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:56:19.422992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:56:19.422992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:56:19.422992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:56:19.422992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:56:19.422992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:56:19.422992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:56:19.422992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:56:19.422992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:56:19.424127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:56:19.424127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:56:19.424127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:56:19.424127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:56:19.424127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:56:19.424127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:56:19.424127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:56:19.424127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:56:19.424127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:56:19.424127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:56:19.424127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:56:19.424127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:56:19.424127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:56:19.424127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:56:19.424127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:56:19.424127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:56:19.425304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:56:19.425304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:56:19.425304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:56:19.425304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:56:19.425304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:56:19.425304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:56:19.425304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:56:19.425304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:56:19.425304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:56:19.425304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:56:19.425304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:56:19.425304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:56:19.425304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 600/1188] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:56:19 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:56:19.433906 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b17746-2c5c-4d5a-904d-57aab161aa0c None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:19.475416 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:19.541840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b17746-2c5c-4d5a-904d-57aab161aa0c None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:19.578093 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197a256-9345-49a5-823a-aae14679a903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:19.578759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197a256-9345-49a5-823a-aae14679a903 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:19.579052 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197a256-9345-49a5-823a-aae14679a903 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:19.579334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0197a256-9345-49a5-823a-aae14679a903 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:19.580115 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197a256-9345-49a5-823a-aae14679a903 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:56:19.580434 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197a256-9345-49a5-823a-aae14679a903 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:19.581822 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0197a256-9345-49a5-823a-aae14679a903 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:19.590017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 589/1189] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:56:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:19.596922 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2693b-a7b2-4608-a71b-8901df9de989 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:19.703734 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f2693b-a7b2-4608-a71b-8901df9de989 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:19.716487 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:19.748535 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db37f5f-96b9-4c04-b6bd-88b182b69a88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:19.749412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db37f5f-96b9-4c04-b6bd-88b182b69a88 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:19.749703 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db37f5f-96b9-4c04-b6bd-88b182b69a88 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:19.749980 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db37f5f-96b9-4c04-b6bd-88b182b69a88 None admin] PATH_INFO: `/v3/users/57efc43eb6e5468080c26a8cc6f714f8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:19.751230 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db37f5f-96b9-4c04-b6bd-88b182b69a88 None admin] RBAC: Authorizing `identity:get_user(user_id=57efc43eb6e5468080c26a8cc6f714f8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:19.752811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db37f5f-96b9-4c04-b6bd-88b182b69a88 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:19.754460 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 601/1190] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:56:19 2022] GET /identity/v3/users/57efc43eb6e5468080c26a8cc6f714f8 => generated 311 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:19.760311 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197a256-9345-49a5-823a-aae14679a903 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:19.874445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:19.878747 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0197a256-9345-49a5-823a-aae14679a903 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:19.912953 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50a0b037-150b-46ad-93a3-72ae02b746ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:19.913816 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0b037-150b-46ad-93a3-72ae02b746ec None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:19.914097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0b037-150b-46ad-93a3-72ae02b746ec None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:19.914425 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50a0b037-150b-46ad-93a3-72ae02b746ec None admin] PATH_INFO: `/v3/projects/31efd53599324dea97d78801918731f0` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:19.916185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a0b037-150b-46ad-93a3-72ae02b746ec None admin] RBAC: Authorizing `identity:get_project(project_id=31efd53599324dea97d78801918731f0)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:19.918213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50a0b037-150b-46ad-93a3-72ae02b746ec None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:19.920516 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 590/1191] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:56:19 2022] GET /identity/v3/projects/31efd53599324dea97d78801918731f0 => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:19.927990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db37f5f-96b9-4c04-b6bd-88b182b69a88 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:20.039478 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db37f5f-96b9-4c04-b6bd-88b182b69a88 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:20.084887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59b482e-883d-4eda-9225-85adf37e1e08 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:20.086277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b482e-883d-4eda-9225-85adf37e1e08 None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:20.086722 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b482e-883d-4eda-9225-85adf37e1e08 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:20.087163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59b482e-883d-4eda-9225-85adf37e1e08 None admin] PATH_INFO: `/v3/projects/31efd53599324dea97d78801918731f0/users/57efc43eb6e5468080c26a8cc6f714f8/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:20.090621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b482e-883d-4eda-9225-85adf37e1e08 None admin] RBAC: Authorizing `identity:create_grant(project_id=31efd53599324dea97d78801918731f0, user_id=57efc43eb6e5468080c26a8cc6f714f8, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:20.093022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59b482e-883d-4eda-9225-85adf37e1e08 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:20.111864 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 602/1192] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:56:19 2022] PUT /identity/v3/projects/31efd53599324dea97d78801918731f0/users/57efc43eb6e5468080c26a8cc6f714f8/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:56:22.269922 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c92515-45c8-4b12-b89f-809b44856deb None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:22.270508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c92515-45c8-4b12-b89f-809b44856deb None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:22.270508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c92515-45c8-4b12-b89f-809b44856deb None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:22.271861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 591/1193] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:56:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:56:22.278464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c37ca-1e2b-430f-bfc5-ac2dd7a86e68 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:22.279186 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c37ca-1e2b-430f-bfc5-ac2dd7a86e68 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:22.279341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc5c37ca-1e2b-430f-bfc5-ac2dd7a86e68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:22.302862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fc5c37ca-1e2b-430f-bfc5-ac2dd7a86e68 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:56:22.304124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fc5c37ca-1e2b-430f-bfc5-ac2dd7a86e68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:56:22.343053 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:22.377492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 603/1194] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:56:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:56:22.402884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c92515-45c8-4b12-b89f-809b44856deb None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:22.519068 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c92515-45c8-4b12-b89f-809b44856deb None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:22.553984 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf0a89-d864-40ed-82af-673146e7c312 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:22.554736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf0a89-d864-40ed-82af-673146e7c312 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:22.555067 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf0a89-d864-40ed-82af-673146e7c312 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:22.555350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bcf0a89-d864-40ed-82af-673146e7c312 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:22.565912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bcf0a89-d864-40ed-82af-673146e7c312 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:56:22.573606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-6bcf0a89-d864-40ed-82af-673146e7c312 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:56:22.573606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:56:22.573606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:56:22.573606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:56:22.573606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:56:22.573606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:56:22.573606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:56:22.573606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:56:22.573606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:56:22.573606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:56:22.573606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:56:22.573606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:56:22.573606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:56:22.573606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:56:22.573606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:56:22.573606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:56:22.574727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:56:22.574727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:56:22.574727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:56:22.574727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:56:22.574727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:56:22.574727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:56:22.574727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:56:22.574727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:56:22.574727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:56:22.574727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:56:22.574727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:56:22.574727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:56:22.574727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:56:22.574727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:56:22.574727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:56:22.574727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:56:22.576354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:56:22.576354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:56:22.576354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:56:22.576354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:56:22.576354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:56:22.576354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:56:22.576354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:56:22.576354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:56:22.576354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:56:22.576354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:56:22.576354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:56:22.576354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:56:22.576354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 592/1195] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:56:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:56:22.584367 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c37ca-1e2b-430f-bfc5-ac2dd7a86e68 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:22.687551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc5c37ca-1e2b-430f-bfc5-ac2dd7a86e68 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:22.722810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b3c94-7b9a-4ce1-b3fd-0386e0851eed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:22.723456 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b3c94-7b9a-4ce1-b3fd-0386e0851eed None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:22.723741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b3c94-7b9a-4ce1-b3fd-0386e0851eed None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:22.724055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-088b3c94-7b9a-4ce1-b3fd-0386e0851eed None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:22.724800 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b3c94-7b9a-4ce1-b3fd-0386e0851eed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:56:22.725110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b3c94-7b9a-4ce1-b3fd-0386e0851eed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:22.726484 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-088b3c94-7b9a-4ce1-b3fd-0386e0851eed None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:22.734560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 604/1196] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:56:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:22.740993 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf0a89-d864-40ed-82af-673146e7c312 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:22.783522 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:22.859534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bcf0a89-d864-40ed-82af-673146e7c312 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:22.896732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac748ac-fd08-4737-b444-d27a1fe4ee09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:22.897609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac748ac-fd08-4737-b444-d27a1fe4ee09 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:22.897894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac748ac-fd08-4737-b444-d27a1fe4ee09 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:22.898175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fac748ac-fd08-4737-b444-d27a1fe4ee09 None admin] PATH_INFO: `/v3/users/57efc43eb6e5468080c26a8cc6f714f8` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:22.899446 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac748ac-fd08-4737-b444-d27a1fe4ee09 None admin] RBAC: Authorizing `identity:get_user(user_id=57efc43eb6e5468080c26a8cc6f714f8)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:22.900928 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fac748ac-fd08-4737-b444-d27a1fe4ee09 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:22.902567 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 593/1197] 10.222.0.85 () {56 vars in 1177 bytes} [Wed May 25 15:56:22 2022] GET /identity/v3/users/57efc43eb6e5468080c26a8cc6f714f8 => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:22.911707 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b3c94-7b9a-4ce1-b3fd-0386e0851eed None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:22.948719 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:23.023014 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-088b3c94-7b9a-4ce1-b3fd-0386e0851eed None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:23.058481 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d67a1-b12d-4729-98d6-c7671d8785dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:23.059125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d67a1-b12d-4729-98d6-c7671d8785dc None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:23.059416 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d67a1-b12d-4729-98d6-c7671d8785dc None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:23.059712 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559d67a1-b12d-4729-98d6-c7671d8785dc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:23.136889 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d67a1-b12d-4729-98d6-c7671d8785dc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:23.138351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559d67a1-b12d-4729-98d6-c7671d8785dc None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:23.168199 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:23.238207 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 605/1198] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:56:22 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:56:23.245059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac748ac-fd08-4737-b444-d27a1fe4ee09 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:23.347674 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:23.355141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fac748ac-fd08-4737-b444-d27a1fe4ee09 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:23.389626 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1c6534-8bc3-42b1-afad-cfec1735a69c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:23.390436 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c6534-8bc3-42b1-afad-cfec1735a69c None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:23.390727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c6534-8bc3-42b1-afad-cfec1735a69c None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:23.391010 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1c6534-8bc3-42b1-afad-cfec1735a69c None admin] PATH_INFO: `/v3/projects/31efd53599324dea97d78801918731f0` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:23.392269 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c6534-8bc3-42b1-afad-cfec1735a69c None admin] RBAC: Authorizing `identity:get_project(project_id=31efd53599324dea97d78801918731f0)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:23.393732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1c6534-8bc3-42b1-afad-cfec1735a69c None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:23.395445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 594/1199] 10.222.0.85 () {56 vars in 1186 bytes} [Wed May 25 15:56:23 2022] GET /identity/v3/projects/31efd53599324dea97d78801918731f0 => generated 361 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:23.401982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d67a1-b12d-4729-98d6-c7671d8785dc None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:56:23.511287 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559d67a1-b12d-4729-98d6-c7671d8785dc None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:56:23.542469 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:23.549163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738b088d-e57a-46d0-9ba3-2d431323d63d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:56:23.549837 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b088d-e57a-46d0-9ba3-2d431323d63d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:56:23.550127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b088d-e57a-46d0-9ba3-2d431323d63d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:56:23.550403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738b088d-e57a-46d0-9ba3-2d431323d63d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:56:23.551255 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b088d-e57a-46d0-9ba3-2d431323d63d None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=31efd53599324dea97d78801918731f0, user.id=57efc43eb6e5468080c26a8cc6f714f8) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:56:23.551574 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b088d-e57a-46d0-9ba3-2d431323d63d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:56:23.553095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738b088d-e57a-46d0-9ba3-2d431323d63d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:56:23.559316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 606/1200] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:56:23 2022] GET /identity/v3/role_assignments?user.id=57efc43eb6e5468080c26a8cc6f714f8&scope.project.id=31efd53599324dea97d78801918731f0&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:56:33.385457 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=19 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:56:33.557083 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:23.853138 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a712b-6034-4b8a-acd9-c2763fe1427f None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:23.853138 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a712b-6034-4b8a-acd9-c2763fe1427f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:23.853138 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90a712b-6034-4b8a-acd9-c2763fe1427f None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:23.853138 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 595/1201] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:57:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:57:23.858912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de9c90-aa5f-49d8-9c1b-aed612e22624 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:23.859378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de9c90-aa5f-49d8-9c1b-aed612e22624 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:23.859466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97de9c90-aa5f-49d8-9c1b-aed612e22624 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:23.878538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-97de9c90-aa5f-49d8-9c1b-aed612e22624 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:57:23.879749 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-97de9c90-aa5f-49d8-9c1b-aed612e22624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:57:23.973859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 607/1202] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:57:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) May 25 15:57:24.002674 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a712b-6034-4b8a-acd9-c2763fe1427f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:24.145604 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90a712b-6034-4b8a-acd9-c2763fe1427f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:24.183664 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c414c-b334-47d3-b33d-c855a5809a8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:24.184519 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c414c-b334-47d3-b33d-c855a5809a8a None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:24.184808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c414c-b334-47d3-b33d-c855a5809a8a None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:24.185070 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9c414c-b334-47d3-b33d-c855a5809a8a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:24.196444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c414c-b334-47d3-b33d-c855a5809a8a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:24.198274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9c414c-b334-47d3-b33d-c855a5809a8a None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:24.217594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-7a9c414c-b334-47d3-b33d-c855a5809a8a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:24.217594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:24.217594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:24.217594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:24.217594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:24.217594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:24.217594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:24.217594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:24.217594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:24.217594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:24.217594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:24.217594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:24.217594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:57:24.217594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:57:24.217594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:57:24.218732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:57:24.218732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:24.218732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:24.218732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:24.218732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:24.218732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:24.218732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:24.218732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:24.218732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:57:24.218732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:24.218732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:24.218732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:24.218732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:24.218732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:24.218732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:24.218732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:24.218732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:24.219704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:57:24.219704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:57:24.219704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:24.219704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:57:24.219704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 596/1203] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:24.225461 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de9c90-aa5f-49d8-9c1b-aed612e22624 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:24.348261 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97de9c90-aa5f-49d8-9c1b-aed612e22624 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:24.384950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:24.391759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2835563a-3449-41ab-8182-c1f684260e31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:24.392547 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835563a-3449-41ab-8182-c1f684260e31 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:24.392811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835563a-3449-41ab-8182-c1f684260e31 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:24.393113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2835563a-3449-41ab-8182-c1f684260e31 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:24.393907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2835563a-3449-41ab-8182-c1f684260e31 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:24.394315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2835563a-3449-41ab-8182-c1f684260e31 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:24.395855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2835563a-3449-41ab-8182-c1f684260e31 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:24.413973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 608/1204] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:57:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:24.416997 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c414c-b334-47d3-b33d-c855a5809a8a None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:24.488770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:24.556605 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9c414c-b334-47d3-b33d-c855a5809a8a None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:24.604991 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3a4329-673e-4f7b-acef-33a740fac6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:24.605688 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a4329-673e-4f7b-acef-33a740fac6b3 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:24.605979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a4329-673e-4f7b-acef-33a740fac6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:24.606260 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3a4329-673e-4f7b-acef-33a740fac6b3 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:24.606983 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a4329-673e-4f7b-acef-33a740fac6b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:24.608561 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a3a4329-673e-4f7b-acef-33a740fac6b3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:24.634028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 597/1205] 10.222.0.85 () {60 vars in 1132 bytes} [Wed May 25 15:57:24 2022] POST /identity/v3/users => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:57:27.004745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e19e6-ee6e-4186-a88c-1db756999a2d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:27.005313 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e19e6-ee6e-4186-a88c-1db756999a2d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:27.005603 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4e19e6-ee6e-4186-a88c-1db756999a2d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:27.007123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 609/1206] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:57:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:57:27.012971 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78deb546-e284-45be-9ecb-903e1a000fe8 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:27.013498 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78deb546-e284-45be-9ecb-903e1a000fe8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:27.013607 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78deb546-e284-45be-9ecb-903e1a000fe8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:27.032213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-78deb546-e284-45be-9ecb-903e1a000fe8 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:57:27.033423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-78deb546-e284-45be-9ecb-903e1a000fe8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:57:27.120032 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 598/1207] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:57:27 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:57:27.141328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e19e6-ee6e-4186-a88c-1db756999a2d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:27.258981 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4e19e6-ee6e-4186-a88c-1db756999a2d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:27.305934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ead3e-87c0-4fe4-a4ec-1e846aab39ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:27.306900 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ead3e-87c0-4fe4-a4ec-1e846aab39ef None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:27.307243 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ead3e-87c0-4fe4-a4ec-1e846aab39ef None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:27.307566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508ead3e-87c0-4fe4-a4ec-1e846aab39ef None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:27.320922 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-508ead3e-87c0-4fe4-a4ec-1e846aab39ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:57:27.329930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-508ead3e-87c0-4fe4-a4ec-1e846aab39ef None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:57:27.329930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:27.329930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:27.329930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:27.329930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:27.329930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:27.329930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:27.329930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:27.329930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:27.329930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:27.329930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:27.329930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:27.329930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:57:27.329930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:57:27.329930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:57:27.329930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:57:27.331070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:57:27.331070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:57:27.331070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:57:27.331070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:57:27.331070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:27.331070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:27.331070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:27.331070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:27.331070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:27.331070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:27.331070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:27.331070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:57:27.331070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:27.331070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:27.331070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:27.331070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:27.332316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:27.332316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:27.332316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:27.332316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:27.332316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:57:27.332316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:57:27.332316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:57:27.332316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:57:27.332316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:57:27.332316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:57:27.332316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:57:27.332316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:27.332316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 610/1208] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:57:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:27.337671 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78deb546-e284-45be-9ecb-903e1a000fe8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:27.380707 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:27.457655 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78deb546-e284-45be-9ecb-903e1a000fe8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:27.502036 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb9e58-680d-4ba4-a3d9-8b4259b68736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:27.503429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb9e58-680d-4ba4-a3d9-8b4259b68736 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:27.503429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb9e58-680d-4ba4-a3d9-8b4259b68736 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:27.503632 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9fb9e58-680d-4ba4-a3d9-8b4259b68736 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:27.504230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb9e58-680d-4ba4-a3d9-8b4259b68736 None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:27.504546 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb9e58-680d-4ba4-a3d9-8b4259b68736 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:27.505992 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9fb9e58-680d-4ba4-a3d9-8b4259b68736 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:27.515955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 599/1209] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:57:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:27.522648 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ead3e-87c0-4fe4-a4ec-1e846aab39ef None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:27.584957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:27.640427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508ead3e-87c0-4fe4-a4ec-1e846aab39ef None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:27.683652 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f59245-3dc6-4864-8579-01285a4a3459 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:27.684581 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f59245-3dc6-4864-8579-01285a4a3459 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:27.684879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f59245-3dc6-4864-8579-01285a4a3459 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:27.685145 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f59245-3dc6-4864-8579-01285a4a3459 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:27.697930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f59245-3dc6-4864-8579-01285a4a3459 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:27.699598 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18f59245-3dc6-4864-8579-01285a4a3459 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:27.718373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-18f59245-3dc6-4864-8579-01285a4a3459 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:27.718373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:27.718373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:27.718373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:27.718373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:27.718373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:27.718373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:27.718373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:27.718373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:27.718373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:27.718373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:27.718373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:27.718373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:57:27.718373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:57:27.718373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:57:27.719539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:57:27.719539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:27.719539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:27.719539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:27.719539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:27.719539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:27.719539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:27.719539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:27.719539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:57:27.719539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:27.719539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:27.719539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:27.719539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:27.719539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:27.719539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:27.719539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:27.719539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:27.720490 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:57:27.720490 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:57:27.720490 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:27.720490 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:27.724814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 611/1210] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:27.725883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb9e58-680d-4ba4-a3d9-8b4259b68736 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:27.832878 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:27.834862 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9fb9e58-680d-4ba4-a3d9-8b4259b68736 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:27.871669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9445a5-36b1-4113-9f69-84ca19e9a3d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:27.872376 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9445a5-36b1-4113-9f69-84ca19e9a3d5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:27.872707 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9445a5-36b1-4113-9f69-84ca19e9a3d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:27.872947 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9445a5-36b1-4113-9f69-84ca19e9a3d5 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:27.873674 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9445a5-36b1-4113-9f69-84ca19e9a3d5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:27.873990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9445a5-36b1-4113-9f69-84ca19e9a3d5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:27.875376 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9445a5-36b1-4113-9f69-84ca19e9a3d5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:27.888092 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 600/1211] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:57:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:57:27.895017 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f59245-3dc6-4864-8579-01285a4a3459 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:28.003240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f59245-3dc6-4864-8579-01285a4a3459 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:28.004971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:28.041778 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05563421-8513-45d1-9531-071915fbaba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:28.042595 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05563421-8513-45d1-9531-071915fbaba0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:28.042889 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05563421-8513-45d1-9531-071915fbaba0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:28.043166 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05563421-8513-45d1-9531-071915fbaba0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:28.051719 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05563421-8513-45d1-9531-071915fbaba0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:28.053469 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05563421-8513-45d1-9531-071915fbaba0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:28.070112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-05563421-8513-45d1-9531-071915fbaba0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 15:57:28.070112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:28.070112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:28.070112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:28.070112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:28.070112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:28.070112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:28.070112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:28.070112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:28.070112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:28.070112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:28.070112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:28.070112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:57:28.070112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:57:28.070112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:57:28.070112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:28.071139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:28.072515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:28.072515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:28.072515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:57:28.072515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:57:28.072515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:57:28.072515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:57:28.072515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:57:28.072515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:57:28.072515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 15:57:28.072515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:28.074454 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 612/1212] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:27 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:28.080364 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9445a5-36b1-4113-9f69-84ca19e9a3d5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:28.188853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9445a5-36b1-4113-9f69-84ca19e9a3d5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:28.231224 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571100a-817f-47da-80a1-c6cedcf98d6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:28.231942 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571100a-817f-47da-80a1-c6cedcf98d6e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:28.232273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571100a-817f-47da-80a1-c6cedcf98d6e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:28.232555 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5571100a-817f-47da-80a1-c6cedcf98d6e None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:28.233394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571100a-817f-47da-80a1-c6cedcf98d6e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:28.233711 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571100a-817f-47da-80a1-c6cedcf98d6e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:28.235313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5571100a-817f-47da-80a1-c6cedcf98d6e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:28.259192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 601/1213] 10.222.0.85 () {56 vars in 1143 bytes} [Wed May 25 15:57:28 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:57:28.261339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05563421-8513-45d1-9531-071915fbaba0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:28.390085 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05563421-8513-45d1-9531-071915fbaba0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:28.432517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21708902-f70a-4ced-be94-31a00e08c5a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:28.433222 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21708902-f70a-4ced-be94-31a00e08c5a2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:28.433530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21708902-f70a-4ced-be94-31a00e08c5a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:28.433801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21708902-f70a-4ced-be94-31a00e08c5a2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:28.473868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:28.516675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21708902-f70a-4ced-be94-31a00e08c5a2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:28.518187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21708902-f70a-4ced-be94-31a00e08c5a2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:28.637051 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 613/1214] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:57:28 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:57:28.643176 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571100a-817f-47da-80a1-c6cedcf98d6e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:28.667724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:28.761686 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5571100a-817f-47da-80a1-c6cedcf98d6e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:28.799839 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb067302-2e9d-43c8-8acf-fa489e5dc0ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:28.800733 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb067302-2e9d-43c8-8acf-fa489e5dc0ed None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:28.801063 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb067302-2e9d-43c8-8acf-fa489e5dc0ed None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:28.801383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb067302-2e9d-43c8-8acf-fa489e5dc0ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:28.813235 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb067302-2e9d-43c8-8acf-fa489e5dc0ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:28.815117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb067302-2e9d-43c8-8acf-fa489e5dc0ed None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:28.834798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-fb067302-2e9d-43c8-8acf-fa489e5dc0ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:28.834798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:28.834798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:28.834798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:28.834798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:28.834798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:28.834798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:28.834798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:28.834798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:28.834798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:28.834798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:28.834798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:28.834798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:57:28.834798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:57:28.834798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:57:28.835877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:57:28.835877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:28.835877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:28.835877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:28.835877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:28.835877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:28.835877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:28.835877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:28.835877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:57:28.835877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:28.835877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:28.835877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:28.835877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:28.835877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:28.835877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:28.835877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:28.835877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:28.837208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:57:28.837208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:57:28.837208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:28.837208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:57:28.837208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 602/1215] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:28.842291 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21708902-f70a-4ced-be94-31a00e08c5a2 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:28.918912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:28.969675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21708902-f70a-4ced-be94-31a00e08c5a2 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:29.012269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20190e23-6c4d-4515-8edd-17ac2f015936 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:29.012915 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20190e23-6c4d-4515-8edd-17ac2f015936 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:29.013078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20190e23-6c4d-4515-8edd-17ac2f015936 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:29.013407 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20190e23-6c4d-4515-8edd-17ac2f015936 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:29.014198 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20190e23-6c4d-4515-8edd-17ac2f015936 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:29.014478 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20190e23-6c4d-4515-8edd-17ac2f015936 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:29.016020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20190e23-6c4d-4515-8edd-17ac2f015936 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:29.029704 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 614/1216] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:57:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:29.036897 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb067302-2e9d-43c8-8acf-fa489e5dc0ed None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:29.122440 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:29.162977 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb067302-2e9d-43c8-8acf-fa489e5dc0ed None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:29.200751 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c4a97-1b27-4bed-96f9-8ebdd44de645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:29.201558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c4a97-1b27-4bed-96f9-8ebdd44de645 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:29.201840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c4a97-1b27-4bed-96f9-8ebdd44de645 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:29.202113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c4a97-1b27-4bed-96f9-8ebdd44de645 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:29.208873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c4a97-1b27-4bed-96f9-8ebdd44de645 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:29.210312 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c4a97-1b27-4bed-96f9-8ebdd44de645 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:29.225072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-523c4a97-1b27-4bed-96f9-8ebdd44de645 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:57:29.225072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:29.225072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:29.225072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:29.225072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:29.225072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:29.225072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:29.225072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:29.225072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:29.225072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:29.225072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:29.225072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:29.225072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:57:29.225072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:57:29.225072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:57:29.226062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:57:29.226062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:29.226062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:29.226062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:29.226062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:29.226062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:29.226062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:29.226062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:29.226062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:57:29.226062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:29.226062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:29.226062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:29.226062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:29.226062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:29.226062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:29.226062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:29.226062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:29.232258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:57:29.232258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:57:29.232258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:57:29.232258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:57:29.232258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:57:29.232258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:57:29.232258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:57:29.232258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:57:29.232258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 603/1217] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:57:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:29.234073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20190e23-6c4d-4515-8edd-17ac2f015936 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:29.355499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20190e23-6c4d-4515-8edd-17ac2f015936 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:29.381963 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:29.396483 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96eb0c1-0b39-4ea7-88c6-45c7aad1cce2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:29.397163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96eb0c1-0b39-4ea7-88c6-45c7aad1cce2 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:29.397486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96eb0c1-0b39-4ea7-88c6-45c7aad1cce2 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:29.397734 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96eb0c1-0b39-4ea7-88c6-45c7aad1cce2 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:29.398642 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96eb0c1-0b39-4ea7-88c6-45c7aad1cce2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:29.398999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96eb0c1-0b39-4ea7-88c6-45c7aad1cce2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:29.400747 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96eb0c1-0b39-4ea7-88c6-45c7aad1cce2 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:29.420065 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 615/1218] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:57:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:29.421407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c4a97-1b27-4bed-96f9-8ebdd44de645 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:29.527023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c4a97-1b27-4bed-96f9-8ebdd44de645 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:29.558938 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:29.567004 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1625e66-fa2e-4f88-9281-8ebd87a7ca14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:29.567895 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1625e66-fa2e-4f88-9281-8ebd87a7ca14 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:29.568238 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1625e66-fa2e-4f88-9281-8ebd87a7ca14 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:29.568540 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1625e66-fa2e-4f88-9281-8ebd87a7ca14 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:29.569461 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1625e66-fa2e-4f88-9281-8ebd87a7ca14 None admin] RBAC: Adding query filter params (role.id=83bf7d9c29d4464dab8030f6d706cd6d, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=82b738f1b8c64590955a640c461b4ef9) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:29.569817 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1625e66-fa2e-4f88-9281-8ebd87a7ca14 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:29.571322 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1625e66-fa2e-4f88-9281-8ebd87a7ca14 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:29.578323 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 604/1219] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:57:29 2022] GET /identity/v3/role_assignments?user.id=82b738f1b8c64590955a640c461b4ef9&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=83bf7d9c29d4464dab8030f6d706cd6d => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:31.813223 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bfa1d-63e8-49db-b9b4-844d1ac6d99b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:31.813768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bfa1d-63e8-49db-b9b4-844d1ac6d99b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:31.813768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bfa1d-63e8-49db-b9b4-844d1ac6d99b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:31.815213 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 616/1220] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:57:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:57:31.820889 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31012df-643f-42c2-8786-0ed84b506dd3 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:31.821705 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31012df-643f-42c2-8786-0ed84b506dd3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:31.821796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31012df-643f-42c2-8786-0ed84b506dd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:31.840144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b31012df-643f-42c2-8786-0ed84b506dd3 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:57:31.841507 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b31012df-643f-42c2-8786-0ed84b506dd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:57:31.914574 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 605/1221] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:57:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:57:31.938106 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bfa1d-63e8-49db-b9b4-844d1ac6d99b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:32.055276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bfa1d-63e8-49db-b9b4-844d1ac6d99b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:32.096576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7afe61-3a5d-4182-a010-21030e654192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:32.097437 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7afe61-3a5d-4182-a010-21030e654192 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:32.097725 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7afe61-3a5d-4182-a010-21030e654192 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:32.098005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7afe61-3a5d-4182-a010-21030e654192 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:32.109461 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7afe61-3a5d-4182-a010-21030e654192 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:57:32.117165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-aa7afe61-3a5d-4182-a010-21030e654192 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:57:32.117165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:32.117165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:32.117165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:32.117165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:32.117165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:32.117165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:32.117165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:32.117165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:32.117165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:32.117165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:32.117165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:32.117165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:57:32.117165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:57:32.117165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:57:32.117165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:57:32.118330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:57:32.118330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:57:32.118330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:57:32.118330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:57:32.118330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:32.118330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:32.118330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:32.118330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:32.118330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:32.118330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:32.118330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:32.118330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:57:32.118330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:32.118330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:32.118330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:32.118330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:32.119206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:32.119206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:32.119206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:32.119206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:32.119206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:57:32.119206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:57:32.119206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:57:32.119206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:57:32.119206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:57:32.119206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:57:32.119206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:57:32.119206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:32.119206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 617/1222] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:57:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:32.124719 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31012df-643f-42c2-8786-0ed84b506dd3 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:32.228069 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31012df-643f-42c2-8786-0ed84b506dd3 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:32.251561 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:32.262506 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad823dae-9dcb-445f-bfcc-088fdc4f173b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:32.263179 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad823dae-9dcb-445f-bfcc-088fdc4f173b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:32.263465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad823dae-9dcb-445f-bfcc-088fdc4f173b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:32.263763 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad823dae-9dcb-445f-bfcc-088fdc4f173b None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:32.264526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad823dae-9dcb-445f-bfcc-088fdc4f173b None admin] RBAC: Adding query filter params (name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:32.264835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad823dae-9dcb-445f-bfcc-088fdc4f173b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:32.266213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad823dae-9dcb-445f-bfcc-088fdc4f173b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:32.274135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 606/1223] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:57:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:32.280297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7afe61-3a5d-4182-a010-21030e654192 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:32.332963 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:32.400105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7afe61-3a5d-4182-a010-21030e654192 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:32.436857 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83536449-da78-4348-bd20-594ccc7e65cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:32.437654 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83536449-da78-4348-bd20-594ccc7e65cf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:32.437938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83536449-da78-4348-bd20-594ccc7e65cf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:32.438217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83536449-da78-4348-bd20-594ccc7e65cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:32.449412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83536449-da78-4348-bd20-594ccc7e65cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:32.450937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83536449-da78-4348-bd20-594ccc7e65cf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:32.468657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-83536449-da78-4348-bd20-594ccc7e65cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:32.468657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:32.468657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:32.468657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:32.468657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:32.468657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:32.468657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:32.468657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:32.468657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:32.468657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:32.468657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:32.468657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:32.468657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:57:32.468657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:57:32.468657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:57:32.469721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:57:32.469721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:32.469721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:32.469721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:32.469721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:32.469721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:32.469721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:32.469721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:32.469721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:57:32.469721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:32.469721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:32.469721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:32.469721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:32.469721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:32.469721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:32.469721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:32.469721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:32.470907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:57:32.470907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:57:32.470907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:32.470907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:32.470907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 618/1224] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:57:32.476188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad823dae-9dcb-445f-bfcc-088fdc4f173b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:32.579574 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad823dae-9dcb-445f-bfcc-088fdc4f173b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:32.614879 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c04fe3-e52d-4220-8a63-caf19e6708e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:32.615566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c04fe3-e52d-4220-8a63-caf19e6708e3 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:32.615883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c04fe3-e52d-4220-8a63-caf19e6708e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:32.616173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c04fe3-e52d-4220-8a63-caf19e6708e3 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:32.616905 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c04fe3-e52d-4220-8a63-caf19e6708e3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:32.617226 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c04fe3-e52d-4220-8a63-caf19e6708e3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:32.618618 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c04fe3-e52d-4220-8a63-caf19e6708e3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:32.630969 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 607/1225] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:57:32.637133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83536449-da78-4348-bd20-594ccc7e65cf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:32.759783 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:32.766966 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83536449-da78-4348-bd20-594ccc7e65cf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:32.802192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9fa4b1-642f-4b29-831a-46bc2b873d75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:32.803007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fa4b1-642f-4b29-831a-46bc2b873d75 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:32.803293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fa4b1-642f-4b29-831a-46bc2b873d75 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:32.803574 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9fa4b1-642f-4b29-831a-46bc2b873d75 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:32.811075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9fa4b1-642f-4b29-831a-46bc2b873d75 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:32.812574 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d9fa4b1-642f-4b29-831a-46bc2b873d75 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:32.827060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-3d9fa4b1-642f-4b29-831a-46bc2b873d75 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 15:57:32.827060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:32.827060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:32.827060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:32.827060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:32.827060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:32.827060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:32.827060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:32.827060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:32.827060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:32.827060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:32.827060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:32.827060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:57:32.827060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:57:32.827060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:57:32.827060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:32.828649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:32.829612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:32.829612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:32.829612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:57:32.829612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:57:32.829612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:57:32.829612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:57:32.829612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:57:32.829612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:57:32.829612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 15:57:32.829612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:32.829612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 619/1226] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:32 2022] GET /identity/v3/users/placement => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:32.834552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c04fe3-e52d-4220-8a63-caf19e6708e3 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:32.886208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:32.941508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c04fe3-e52d-4220-8a63-caf19e6708e3 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:32.978702 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721da7d4-8780-4a74-8b9c-e53dae45cbe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:32.979388 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721da7d4-8780-4a74-8b9c-e53dae45cbe8 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:32.979678 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721da7d4-8780-4a74-8b9c-e53dae45cbe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:32.979988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721da7d4-8780-4a74-8b9c-e53dae45cbe8 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:32.980838 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721da7d4-8780-4a74-8b9c-e53dae45cbe8 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:32.981155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721da7d4-8780-4a74-8b9c-e53dae45cbe8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:32.982604 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721da7d4-8780-4a74-8b9c-e53dae45cbe8 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:33.000182 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 608/1227] 10.222.0.85 () {56 vars in 1143 bytes} [Wed May 25 15:57:32 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:33.005974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9fa4b1-642f-4b29-831a-46bc2b873d75 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:33.119106 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9fa4b1-642f-4b29-831a-46bc2b873d75 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:33.156331 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1832d-021f-4f1a-b132-83f63a609f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:33.157158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1832d-021f-4f1a-b132-83f63a609f35 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:33.157416 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1832d-021f-4f1a-b132-83f63a609f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:33.157690 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09d1832d-021f-4f1a-b132-83f63a609f35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:33.169031 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1832d-021f-4f1a-b132-83f63a609f35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:33.170517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09d1832d-021f-4f1a-b132-83f63a609f35 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:33.188019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-09d1832d-021f-4f1a-b132-83f63a609f35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:33.188019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:33.188019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:33.188019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:33.188019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:33.188019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:33.188019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:33.188019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:33.188019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:33.188019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:33.188019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:33.188019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:33.188019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:57:33.188019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:57:33.188019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:57:33.189002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:57:33.189002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:33.189002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:33.189002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:33.189002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:33.189002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:33.189002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:33.189002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:33.189002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:57:33.189002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:33.189002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:33.189002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:33.189002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:33.189002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:33.189002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:33.189002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:33.189002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:33.189915 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:57:33.189915 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:57:33.189915 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:33.189915 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:33.189915 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 620/1228] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:33.195301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721da7d4-8780-4a74-8b9c-e53dae45cbe8 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:33.305270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721da7d4-8780-4a74-8b9c-e53dae45cbe8 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:33.307667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:33.341159 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b62b42-1049-4abe-9bcc-0188be75c91b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:33.341845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b62b42-1049-4abe-9bcc-0188be75c91b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:33.342142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b62b42-1049-4abe-9bcc-0188be75c91b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:33.342433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b62b42-1049-4abe-9bcc-0188be75c91b None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:33.343233 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b62b42-1049-4abe-9bcc-0188be75c91b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:33.343551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b62b42-1049-4abe-9bcc-0188be75c91b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:33.344976 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b62b42-1049-4abe-9bcc-0188be75c91b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:33.360239 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 609/1229] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:57:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:33.366473 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1832d-021f-4f1a-b132-83f63a609f35 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:33.378452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:33.483401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09d1832d-021f-4f1a-b132-83f63a609f35 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:33.520370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed8ae9-5680-40a0-b6cf-86c33d484902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:33.521189 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed8ae9-5680-40a0-b6cf-86c33d484902 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:33.521563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed8ae9-5680-40a0-b6cf-86c33d484902 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:33.521854 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fed8ae9-5680-40a0-b6cf-86c33d484902 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:33.530538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed8ae9-5680-40a0-b6cf-86c33d484902 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:33.532096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fed8ae9-5680-40a0-b6cf-86c33d484902 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:33.548733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-1fed8ae9-5680-40a0-b6cf-86c33d484902 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:57:33.548733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:33.548733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:33.548733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:33.548733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:33.548733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:33.548733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:33.548733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:33.548733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:33.548733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:33.548733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:33.548733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:33.548733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:57:33.548733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:57:33.548733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:57:33.550124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:57:33.550124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:33.550124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:33.550124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:33.550124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:33.550124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:33.550124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:33.550124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:33.550124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:57:33.550124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:33.550124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:33.550124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:33.550124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:33.550124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:33.550124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:33.550124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:33.550124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:33.551105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:57:33.551105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:57:33.551105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:57:33.551105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:57:33.551105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:57:33.551105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:57:33.551105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:57:33.551105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:33.551105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 621/1230] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:57:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:33.556364 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b62b42-1049-4abe-9bcc-0188be75c91b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:33.664608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b62b42-1049-4abe-9bcc-0188be75c91b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:33.698159 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e762b315-427e-4fdd-96bc-384a3e58d986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:33.698839 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762b315-427e-4fdd-96bc-384a3e58d986 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:33.699119 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762b315-427e-4fdd-96bc-384a3e58d986 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:33.699398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e762b315-427e-4fdd-96bc-384a3e58d986 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:33.700231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762b315-427e-4fdd-96bc-384a3e58d986 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:33.700552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762b315-427e-4fdd-96bc-384a3e58d986 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:33.702029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e762b315-427e-4fdd-96bc-384a3e58d986 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:33.714567 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 610/1231] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:57:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:33.720958 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed8ae9-5680-40a0-b6cf-86c33d484902 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:33.796008 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:33.837205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fed8ae9-5680-40a0-b6cf-86c33d484902 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:33.871990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4549d1c-72ca-49d2-b1e6-d529d360f0ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:33.872936 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4549d1c-72ca-49d2-b1e6-d529d360f0ae None admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:33.873229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4549d1c-72ca-49d2-b1e6-d529d360f0ae None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:33.873594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4549d1c-72ca-49d2-b1e6-d529d360f0ae None admin] PATH_INFO: `/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/82b738f1b8c64590955a640c461b4ef9/roles/83bf7d9c29d4464dab8030f6d706cd6d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:33.890463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4549d1c-72ca-49d2-b1e6-d529d360f0ae None admin] RBAC: Authorizing `identity:create_grant(project_id=5584ebdaf133444ca69efc5554bf48e9, user_id=82b738f1b8c64590955a640c461b4ef9, role_id=83bf7d9c29d4464dab8030f6d706cd6d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:33.892153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4549d1c-72ca-49d2-b1e6-d529d360f0ae None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:33.906569 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 622/1232] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:57:33 2022] PUT /identity/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/82b738f1b8c64590955a640c461b4ef9/roles/83bf7d9c29d4464dab8030f6d706cd6d => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:57:36.069877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e83de-f0b1-48b9-81f0-4a2ea28f5c46 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:36.070538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e83de-f0b1-48b9-81f0-4a2ea28f5c46 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:36.070538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06e83de-f0b1-48b9-81f0-4a2ea28f5c46 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:36.072234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 611/1233] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:57:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:57:36.077620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d183f59-03b0-451d-9462-14c51ef19cb4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:36.078109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d183f59-03b0-451d-9462-14c51ef19cb4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:36.078241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d183f59-03b0-451d-9462-14c51ef19cb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:36.095566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1d183f59-03b0-451d-9462-14c51ef19cb4 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:57:36.096889 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1d183f59-03b0-451d-9462-14c51ef19cb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:57:36.170729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 623/1234] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:57:36 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:57:36.194626 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e83de-f0b1-48b9-81f0-4a2ea28f5c46 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:36.221784 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:36.309511 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06e83de-f0b1-48b9-81f0-4a2ea28f5c46 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:36.347302 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1f2ba-4597-4def-a8f7-4bd9614d7048 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:36.348103 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1f2ba-4597-4def-a8f7-4bd9614d7048 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:36.348437 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1f2ba-4597-4def-a8f7-4bd9614d7048 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:36.348784 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c1f2ba-4597-4def-a8f7-4bd9614d7048 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:36.364415 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c1f2ba-4597-4def-a8f7-4bd9614d7048 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:57:36.373220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-22c1f2ba-4597-4def-a8f7-4bd9614d7048 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. May 25 15:57:36.373220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:36.373220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:36.373220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:36.373220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:36.373220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:36.373220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:36.373220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:36.373220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:36.373220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:36.373220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:36.373220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:36.373220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:57:36.373220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:57:36.373220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:57:36.373220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:57:36.374786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:57:36.374786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:57:36.374786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:57:36.374786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:57:36.374786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:36.374786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:36.374786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:36.374786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:36.374786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:36.374786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:36.374786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:36.374786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:57:36.374786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:36.374786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:36.374786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:36.374786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:36.376581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:36.376581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:36.376581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:36.376581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:36.376581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:57:36.376581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:57:36.376581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:57:36.376581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:57:36.376581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:57:36.376581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:57:36.376581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. May 25 15:57:36.376581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:57:36.376581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 612/1235] 10.222.0.85 () {56 vars in 1102 bytes} [Wed May 25 15:57:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:36.381923 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d183f59-03b0-451d-9462-14c51ef19cb4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:36.429318 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:36.486782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d183f59-03b0-451d-9462-14c51ef19cb4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:36.521942 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3146f0e-2786-48df-8eb9-01f5ecac1ee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:36.522724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3146f0e-2786-48df-8eb9-01f5ecac1ee6 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:36.523059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3146f0e-2786-48df-8eb9-01f5ecac1ee6 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:36.523378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3146f0e-2786-48df-8eb9-01f5ecac1ee6 None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:36.524180 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3146f0e-2786-48df-8eb9-01f5ecac1ee6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:36.524500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3146f0e-2786-48df-8eb9-01f5ecac1ee6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:36.525986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3146f0e-2786-48df-8eb9-01f5ecac1ee6 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:36.535943 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 624/1236] 10.222.0.85 () {56 vars in 1103 bytes} [Wed May 25 15:57:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:36.541952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1f2ba-4597-4def-a8f7-4bd9614d7048 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:36.632977 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:36.652250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c1f2ba-4597-4def-a8f7-4bd9614d7048 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:36.694754 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe2db2-5de3-4d28-9ed6-b0e1ccc3976e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:36.695637 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe2db2-5de3-4d28-9ed6-b0e1ccc3976e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:36.696305 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe2db2-5de3-4d28-9ed6-b0e1ccc3976e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:36.696598 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fafe2db2-5de3-4d28-9ed6-b0e1ccc3976e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:36.707910 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe2db2-5de3-4d28-9ed6-b0e1ccc3976e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:36.709552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fafe2db2-5de3-4d28-9ed6-b0e1ccc3976e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:36.731165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-fafe2db2-5de3-4d28-9ed6-b0e1ccc3976e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:36.731165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:36.731165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:36.731165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:36.731165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:36.731165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:36.731165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:36.731165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:36.731165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:36.731165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:36.731165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:36.731165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:36.731165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:57:36.731165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:57:36.731165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:57:36.732488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:57:36.732488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:36.732488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:36.732488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:36.732488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:36.732488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:36.732488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:36.732488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:36.732488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:57:36.732488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:36.732488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:36.732488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:36.732488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:36.732488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:36.732488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:36.732488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:36.732488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:36.733601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:57:36.733601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:57:36.733601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:36.733601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:57:36.737822 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 613/1237] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:57:36.738821 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3146f0e-2786-48df-8eb9-01f5ecac1ee6 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:36.855173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3146f0e-2786-48df-8eb9-01f5ecac1ee6 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:36.865292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:36.896171 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b57b0e-c286-4a7c-90fc-44110fe6ab24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:36.896859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b57b0e-c286-4a7c-90fc-44110fe6ab24 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:36.897177 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b57b0e-c286-4a7c-90fc-44110fe6ab24 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:36.897454 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b57b0e-c286-4a7c-90fc-44110fe6ab24 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:36.898272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b57b0e-c286-4a7c-90fc-44110fe6ab24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:36.898569 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b57b0e-c286-4a7c-90fc-44110fe6ab24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:36.900116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b57b0e-c286-4a7c-90fc-44110fe6ab24 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:36.914347 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 625/1238] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:57:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:36.920303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe2db2-5de3-4d28-9ed6-b0e1ccc3976e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:37.042659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fafe2db2-5de3-4d28-9ed6-b0e1ccc3976e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:37.080868 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e214a-6ae5-4378-9e1d-9e76e2573edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:37.081765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e214a-6ae5-4378-9e1d-9e76e2573edd None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:37.082244 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e214a-6ae5-4378-9e1d-9e76e2573edd None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:37.082589 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8e214a-6ae5-4378-9e1d-9e76e2573edd None admin] PATH_INFO: `/v3/users/placement` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:37.086153 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:37.092195 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e214a-6ae5-4378-9e1d-9e76e2573edd None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:37.093906 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c8e214a-6ae5-4378-9e1d-9e76e2573edd None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:37.109241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-0c8e214a-6ae5-4378-9e1d-9e76e2573edd None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 15:57:37.109241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:37.109241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:37.109241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:37.109241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:37.109241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:37.109241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:37.109241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:37.109241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:37.109241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:37.109241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:37.109241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:37.109241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:57:37.109241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:57:37.109241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:57:37.109241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:37.110473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:37.116207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:37.116207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:37.116207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:57:37.116207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:57:37.116207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:57:37.116207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:57:37.116207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:57:37.116207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:57:37.116207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 15:57:37.116207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:57:37.116207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 614/1239] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:36 2022] GET /identity/v3/users/placement => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:57:37.116997 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b57b0e-c286-4a7c-90fc-44110fe6ab24 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:37.228607 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b57b0e-c286-4a7c-90fc-44110fe6ab24 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:37.267704 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6436ab4-e11f-4751-9ef9-360f2cc5714e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:37.268425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6436ab4-e11f-4751-9ef9-360f2cc5714e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:37.268717 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6436ab4-e11f-4751-9ef9-360f2cc5714e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:37.269000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6436ab4-e11f-4751-9ef9-360f2cc5714e None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:37.269853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6436ab4-e11f-4751-9ef9-360f2cc5714e None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:37.270171 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6436ab4-e11f-4751-9ef9-360f2cc5714e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:37.271685 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6436ab4-e11f-4751-9ef9-360f2cc5714e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:37.296940 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 626/1240] 10.222.0.85 () {56 vars in 1143 bytes} [Wed May 25 15:57:37 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:37.298321 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e214a-6ae5-4378-9e1d-9e76e2573edd None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:37.421144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8e214a-6ae5-4378-9e1d-9e76e2573edd None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:37.459374 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba1373-f915-41a5-b880-1c9ab2180de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:37.460073 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba1373-f915-41a5-b880-1c9ab2180de5 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:37.460362 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba1373-f915-41a5-b880-1c9ab2180de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:37.460649 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba1373-f915-41a5-b880-1c9ab2180de5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:37.528100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:37.535020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba1373-f915-41a5-b880-1c9ab2180de5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:37.536591 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba1373-f915-41a5-b880-1c9ab2180de5 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:37.643195 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 615/1241] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:57:37 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:57:37.650031 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6436ab4-e11f-4751-9ef9-360f2cc5714e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:37.679632 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:37.764712 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6436ab4-e11f-4751-9ef9-360f2cc5714e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:37.802240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97bec9-d5a5-41c2-8784-5eca8fc1e2a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:37.802975 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97bec9-d5a5-41c2-8784-5eca8fc1e2a4 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:37.803272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97bec9-d5a5-41c2-8784-5eca8fc1e2a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:37.803558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb97bec9-d5a5-41c2-8784-5eca8fc1e2a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:37.814239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97bec9-d5a5-41c2-8784-5eca8fc1e2a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:37.815716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb97bec9-d5a5-41c2-8784-5eca8fc1e2a4 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:37.834976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-cb97bec9-d5a5-41c2-8784-5eca8fc1e2a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:37.834976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:37.834976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:37.834976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:37.834976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:37.834976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:37.834976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:37.834976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:37.834976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:37.834976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:37.834976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:37.834976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:37.834976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:57:37.834976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:57:37.834976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:57:37.836698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:57:37.836698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:37.836698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:37.836698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:37.836698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:37.836698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:37.836698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:37.836698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:37.836698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:57:37.836698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:37.836698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:37.836698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:37.836698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:37.836698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:37.836698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:37.836698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:37.836698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:37.837876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:57:37.837876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:57:37.837876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:37.837876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:37.837876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 627/1242] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:37.842141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba1373-f915-41a5-b880-1c9ab2180de5 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:37.940275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:37.947722 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba1373-f915-41a5-b880-1c9ab2180de5 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:37.986380 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a01b80-be8e-4b1e-9847-99e1f7559215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:37.987058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a01b80-be8e-4b1e-9847-99e1f7559215 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:37.987351 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a01b80-be8e-4b1e-9847-99e1f7559215 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:37.987634 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a01b80-be8e-4b1e-9847-99e1f7559215 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:37.988392 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a01b80-be8e-4b1e-9847-99e1f7559215 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:37.988710 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a01b80-be8e-4b1e-9847-99e1f7559215 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:37.990152 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a01b80-be8e-4b1e-9847-99e1f7559215 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:38.001713 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 616/1243] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:57:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:38.008079 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97bec9-d5a5-41c2-8784-5eca8fc1e2a4 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:38.105428 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:38.134020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb97bec9-d5a5-41c2-8784-5eca8fc1e2a4 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:38.171676 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c02c00-d066-475a-be51-e62ee5e1588e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:38.172528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c02c00-d066-475a-be51-e62ee5e1588e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:38.172820 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c02c00-d066-475a-be51-e62ee5e1588e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:38.173107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c02c00-d066-475a-be51-e62ee5e1588e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:38.181327 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c02c00-d066-475a-be51-e62ee5e1588e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:38.183075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c02c00-d066-475a-be51-e62ee5e1588e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:38.199626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-c3c02c00-d066-475a-be51-e62ee5e1588e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:57:38.199626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:38.199626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:38.199626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:38.199626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:38.199626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:38.199626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:38.199626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:38.199626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:38.199626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:38.199626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:38.199626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:38.199626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:57:38.199626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:57:38.199626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:57:38.200718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:57:38.200718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:38.200718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:38.200718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:38.200718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:38.200718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:38.200718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:38.200718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:38.200718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:57:38.200718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:38.200718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:38.200718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:38.200718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:38.200718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:38.200718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:38.200718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:38.200718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:38.202041 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:57:38.202041 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:57:38.202041 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:57:38.202041 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:57:38.202041 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:57:38.202041 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:57:38.202041 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:57:38.202041 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:38.202041 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 628/1244] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:57:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:38.206974 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a01b80-be8e-4b1e-9847-99e1f7559215 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:38.319796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a01b80-be8e-4b1e-9847-99e1f7559215 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:38.361157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a975b5-bbd0-4b7f-b876-7496c30b8e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:38.361805 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a975b5-bbd0-4b7f-b876-7496c30b8e62 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:38.362183 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a975b5-bbd0-4b7f-b876-7496c30b8e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:38.362455 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a975b5-bbd0-4b7f-b876-7496c30b8e62 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:38.363512 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a975b5-bbd0-4b7f-b876-7496c30b8e62 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:38.363799 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a975b5-bbd0-4b7f-b876-7496c30b8e62 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:38.365438 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a975b5-bbd0-4b7f-b876-7496c30b8e62 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:38.378613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 617/1245] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:57:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:38.385645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c02c00-d066-475a-be51-e62ee5e1588e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:38.499272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c02c00-d066-475a-be51-e62ee5e1588e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:38.535231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55835a63-540e-4ec5-8cf0-b7793e61119f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:38.535923 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55835a63-540e-4ec5-8cf0-b7793e61119f None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:38.536272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55835a63-540e-4ec5-8cf0-b7793e61119f None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:38.536556 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55835a63-540e-4ec5-8cf0-b7793e61119f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:38.537410 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55835a63-540e-4ec5-8cf0-b7793e61119f None admin] RBAC: Adding query filter params (role.id=83bf7d9c29d4464dab8030f6d706cd6d, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=82b738f1b8c64590955a640c461b4ef9) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:38.537733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55835a63-540e-4ec5-8cf0-b7793e61119f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:38.539192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55835a63-540e-4ec5-8cf0-b7793e61119f None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:38.545469 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 629/1246] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:57:38 2022] GET /identity/v3/role_assignments?user.id=82b738f1b8c64590955a640c461b4ef9&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=83bf7d9c29d4464dab8030f6d706cd6d => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:40.854761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7710a0e-f94f-4ad1-b81a-6f0c04ed83f1 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:40.855312 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7710a0e-f94f-4ad1-b81a-6f0c04ed83f1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:40.855312 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7710a0e-f94f-4ad1-b81a-6f0c04ed83f1 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:40.856769 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 618/1247] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:57:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:57:40.862529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f9a72-d9cd-4329-a6a7-f3dcbe42dc21 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:40.863114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f9a72-d9cd-4329-a6a7-f3dcbe42dc21 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:40.863214 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548f9a72-d9cd-4329-a6a7-f3dcbe42dc21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:40.868484 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:40.881599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-548f9a72-d9cd-4329-a6a7-f3dcbe42dc21 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:57:40.882848 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-548f9a72-d9cd-4329-a6a7-f3dcbe42dc21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:57:40.956673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 630/1248] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:57:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:57:40.980196 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7710a0e-f94f-4ad1-b81a-6f0c04ed83f1 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:41.001709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:41.113122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7710a0e-f94f-4ad1-b81a-6f0c04ed83f1 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:41.150709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c384ad7-43a8-4cf1-abd7-fe3ac530d643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:41.151606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c384ad7-43a8-4cf1-abd7-fe3ac530d643 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:41.151943 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c384ad7-43a8-4cf1-abd7-fe3ac530d643 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:41.152232 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c384ad7-43a8-4cf1-abd7-fe3ac530d643 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:41.163885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c384ad7-43a8-4cf1-abd7-fe3ac530d643 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:57:41.172350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-7c384ad7-43a8-4cf1-abd7-fe3ac530d643 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:57:41.172350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:41.172350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:41.172350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:41.172350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:41.172350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:41.172350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:41.172350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:41.172350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:41.172350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:41.172350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:41.172350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:41.172350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:57:41.172350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:57:41.172350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:57:41.172350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:57:41.173537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:57:41.173537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:57:41.173537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:57:41.173537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:57:41.173537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:41.173537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:41.173537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:41.173537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:41.173537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:41.173537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:41.173537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:41.173537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:57:41.173537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:41.173537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:41.173537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:41.173537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:41.174581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:41.174581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:41.174581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:41.174581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:41.174581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:57:41.174581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:57:41.174581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:57:41.174581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:57:41.174581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:57:41.174581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:57:41.174581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:57:41.174581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:57:41.174581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 619/1249] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:57:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:41.179983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f9a72-d9cd-4329-a6a7-f3dcbe42dc21 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:41.288606 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548f9a72-d9cd-4329-a6a7-f3dcbe42dc21 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:41.325284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:41.332068 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8f823-95f7-4cde-bfed-63e41a91602e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:41.332806 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8f823-95f7-4cde-bfed-63e41a91602e None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:41.333114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8f823-95f7-4cde-bfed-63e41a91602e None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:41.333459 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4f8f823-95f7-4cde-bfed-63e41a91602e None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:41.334306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8f823-95f7-4cde-bfed-63e41a91602e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:41.334635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8f823-95f7-4cde-bfed-63e41a91602e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:41.336288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4f8f823-95f7-4cde-bfed-63e41a91602e None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:41.345615 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 631/1250] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:57:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:41.351720 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c384ad7-43a8-4cf1-abd7-fe3ac530d643 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:41.442663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:41.468610 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c384ad7-43a8-4cf1-abd7-fe3ac530d643 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:41.504821 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94f1d9-ae18-4da9-9f8f-af339296ad96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:41.505630 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94f1d9-ae18-4da9-9f8f-af339296ad96 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:41.505915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94f1d9-ae18-4da9-9f8f-af339296ad96 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:41.506197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c94f1d9-ae18-4da9-9f8f-af339296ad96 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:41.518527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94f1d9-ae18-4da9-9f8f-af339296ad96 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:41.520260 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c94f1d9-ae18-4da9-9f8f-af339296ad96 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:41.538797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-6c94f1d9-ae18-4da9-9f8f-af339296ad96 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:41.538797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:41.538797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:41.538797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:41.538797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:41.538797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:41.538797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:41.538797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:41.538797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:41.538797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:41.538797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:41.538797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:41.538797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:57:41.538797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:57:41.538797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:57:41.539811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:57:41.539811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:41.539811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:41.539811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:41.539811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:41.539811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:41.539811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:41.539811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:41.539811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:57:41.539811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:41.539811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:41.539811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:41.539811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:41.539811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:41.539811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:41.539811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:41.539811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:41.540766 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:57:41.540766 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:57:41.540766 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:41.540766 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:57:41.540766 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 620/1251] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:41.546204 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8f823-95f7-4cde-bfed-63e41a91602e None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:41.664459 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4f8f823-95f7-4cde-bfed-63e41a91602e None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:41.701337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3084c4b-5df8-4e10-8829-585ec750bd77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:41.701970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3084c4b-5df8-4e10-8829-585ec750bd77 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:41.702260 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3084c4b-5df8-4e10-8829-585ec750bd77 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:41.702543 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3084c4b-5df8-4e10-8829-585ec750bd77 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:41.703273 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3084c4b-5df8-4e10-8829-585ec750bd77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:41.703590 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3084c4b-5df8-4e10-8829-585ec750bd77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:41.705087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3084c4b-5df8-4e10-8829-585ec750bd77 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:41.718087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 632/1252] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:57:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:41.724099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94f1d9-ae18-4da9-9f8f-af339296ad96 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:41.828767 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c94f1d9-ae18-4da9-9f8f-af339296ad96 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:41.856823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:41.865695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1210689-f8f8-4b26-9bfe-4cfd225a018d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:41.866575 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1210689-f8f8-4b26-9bfe-4cfd225a018d None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:41.866907 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1210689-f8f8-4b26-9bfe-4cfd225a018d None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:41.867235 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1210689-f8f8-4b26-9bfe-4cfd225a018d None admin] PATH_INFO: `/v3/users/placement` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:41.875964 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1210689-f8f8-4b26-9bfe-4cfd225a018d None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:41.877570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1210689-f8f8-4b26-9bfe-4cfd225a018d None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:41.892600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-c1210689-f8f8-4b26-9bfe-4cfd225a018d None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 15:57:41.892600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:41.892600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:41.892600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:41.892600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:41.892600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:41.892600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:41.892600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:41.892600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:41.892600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:41.892600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:41.892600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:41.892600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:57:41.892600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:57:41.892600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:57:41.892600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:41.893786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:41.894884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:41.894884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:41.894884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:57:41.894884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:57:41.894884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:57:41.894884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:57:41.894884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:57:41.894884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:57:41.894884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 15:57:41.894884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:57:41.894884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 621/1253] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:41.900206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3084c4b-5df8-4e10-8829-585ec750bd77 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:41.960169 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:42.015255 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3084c4b-5df8-4e10-8829-585ec750bd77 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:42.054941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a990a3-83d9-4f8e-83e0-01a817f25cb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:42.055621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a990a3-83d9-4f8e-83e0-01a817f25cb9 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:42.056020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a990a3-83d9-4f8e-83e0-01a817f25cb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:42.056219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a990a3-83d9-4f8e-83e0-01a817f25cb9 None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:42.057110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a990a3-83d9-4f8e-83e0-01a817f25cb9 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:42.057429 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a990a3-83d9-4f8e-83e0-01a817f25cb9 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:42.059163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a990a3-83d9-4f8e-83e0-01a817f25cb9 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:42.080983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 633/1254] 10.222.0.85 () {56 vars in 1143 bytes} [Wed May 25 15:57:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:42.087236 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1210689-f8f8-4b26-9bfe-4cfd225a018d None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:42.202907 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1210689-f8f8-4b26-9bfe-4cfd225a018d None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:42.237811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf9b46-23dc-476a-a818-043611faf4e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:42.238401 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf9b46-23dc-476a-a818-043611faf4e9 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:42.238698 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf9b46-23dc-476a-a818-043611faf4e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:42.239028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdf9b46-23dc-476a-a818-043611faf4e9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:42.297775 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:42.314747 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf9b46-23dc-476a-a818-043611faf4e9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:42.316310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdf9b46-23dc-476a-a818-043611faf4e9 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:42.419261 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 622/1255] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:57:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:57:42.425327 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a990a3-83d9-4f8e-83e0-01a817f25cb9 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:42.536424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a990a3-83d9-4f8e-83e0-01a817f25cb9 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:42.551965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:42.575095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee37e3b-7c11-4d05-af85-b822469e7b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:42.576011 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee37e3b-7c11-4d05-af85-b822469e7b9c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:42.576303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee37e3b-7c11-4d05-af85-b822469e7b9c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:42.576583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ee37e3b-7c11-4d05-af85-b822469e7b9c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:42.592291 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee37e3b-7c11-4d05-af85-b822469e7b9c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:42.593983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ee37e3b-7c11-4d05-af85-b822469e7b9c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:42.612467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-0ee37e3b-7c11-4d05-af85-b822469e7b9c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:42.612467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:42.612467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:42.612467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:42.612467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:42.612467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:42.612467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:42.612467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:42.612467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:42.612467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:42.612467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:42.612467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:42.612467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:57:42.612467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:57:42.612467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:57:42.613723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:57:42.613723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:42.613723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:42.613723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:42.613723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:42.613723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:42.613723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:42.613723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:42.613723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:57:42.613723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:42.613723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:42.613723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:42.613723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:42.613723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:42.613723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:42.613723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:42.613723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:42.615120 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:57:42.615120 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:57:42.615120 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:42.615120 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:42.615120 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 634/1256] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:42.619800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf9b46-23dc-476a-a818-043611faf4e9 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:42.713088 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:42.728823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdf9b46-23dc-476a-a818-043611faf4e9 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:42.764823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f737945-8f97-4088-905b-788fa87d4bb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:42.765497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f737945-8f97-4088-905b-788fa87d4bb1 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:42.765791 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f737945-8f97-4088-905b-788fa87d4bb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:42.766076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f737945-8f97-4088-905b-788fa87d4bb1 None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:42.766822 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f737945-8f97-4088-905b-788fa87d4bb1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:42.767123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f737945-8f97-4088-905b-788fa87d4bb1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:42.768550 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f737945-8f97-4088-905b-788fa87d4bb1 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:42.780050 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 623/1257] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:57:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:42.785931 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee37e3b-7c11-4d05-af85-b822469e7b9c None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:42.904372 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ee37e3b-7c11-4d05-af85-b822469e7b9c None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:42.944904 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c997c55-7cc9-4a39-884f-f828d724cccf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:42.946248 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c997c55-7cc9-4a39-884f-f828d724cccf None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:42.946616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c997c55-7cc9-4a39-884f-f828d724cccf None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:42.946947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c997c55-7cc9-4a39-884f-f828d724cccf None admin] PATH_INFO: `/v3/projects/service` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:42.954376 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c997c55-7cc9-4a39-884f-f828d724cccf None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:42.956135 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c997c55-7cc9-4a39-884f-f828d724cccf None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:42.971957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-8c997c55-7cc9-4a39-884f-f828d724cccf None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:57:42.971957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:42.971957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:42.971957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:42.971957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:42.971957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:42.971957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:42.971957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:42.971957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:42.971957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:42.971957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:42.971957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:42.971957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:57:42.971957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:57:42.971957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:57:42.973100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:57:42.973100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:42.973100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:42.973100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:42.973100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:42.973100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:42.973100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:42.973100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:42.973100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:57:42.973100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:42.973100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:42.973100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:42.973100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:42.973100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:42.973100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:42.973100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:42.973100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:42.974399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:57:42.974399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:57:42.974399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:57:42.974399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:57:42.974399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:57:42.974399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:57:42.974399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:57:42.974399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:42.974399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 635/1258] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:57:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:42.979376 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f737945-8f97-4088-905b-788fa87d4bb1 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:43.092009 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f737945-8f97-4088-905b-788fa87d4bb1 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:43.131944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e8b27-f573-470b-b788-b6aa883b7b45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:43.132785 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e8b27-f573-470b-b788-b6aa883b7b45 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:43.133133 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e8b27-f573-470b-b788-b6aa883b7b45 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:43.133638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e8b27-f573-470b-b788-b6aa883b7b45 None admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:43.134268 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e8b27-f573-470b-b788-b6aa883b7b45 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:43.134722 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e8b27-f573-470b-b788-b6aa883b7b45 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:43.136515 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e8b27-f573-470b-b788-b6aa883b7b45 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:43.150094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 624/1259] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:57:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:43.156513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c997c55-7cc9-4a39-884f-f828d724cccf None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:43.185383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:43.262808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c997c55-7cc9-4a39-884f-f828d724cccf None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:43.307947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d5ff52-6492-4001-90f2-1c7531613f53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:43.308617 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5ff52-6492-4001-90f2-1c7531613f53 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:43.308905 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5ff52-6492-4001-90f2-1c7531613f53 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:43.309254 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d5ff52-6492-4001-90f2-1c7531613f53 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:43.310117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d5ff52-6492-4001-90f2-1c7531613f53 None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=82b738f1b8c64590955a640c461b4ef9) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:43.310446 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d5ff52-6492-4001-90f2-1c7531613f53 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:43.312064 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d5ff52-6492-4001-90f2-1c7531613f53 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:43.318573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 636/1260] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:57:43 2022] GET /identity/v3/role_assignments?user.id=82b738f1b8c64590955a640c461b4ef9&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 259 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:45.520162 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c481a-3a33-4bba-af48-f32d9a3cac01 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:45.520748 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c481a-3a33-4bba-af48-f32d9a3cac01 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:45.520748 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b1c481a-3a33-4bba-af48-f32d9a3cac01 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:45.522131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 625/1261] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:57:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:57:45.528586 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e810f-14aa-434b-91c1-17a3036d11ad None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:45.529299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e810f-14aa-434b-91c1-17a3036d11ad None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:45.529488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9e810f-14aa-434b-91c1-17a3036d11ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:45.550887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0e9e810f-14aa-434b-91c1-17a3036d11ad None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:57:45.552116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0e9e810f-14aa-434b-91c1-17a3036d11ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:57:45.634482 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 637/1262] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:57:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:57:45.657186 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c481a-3a33-4bba-af48-f32d9a3cac01 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:45.669426 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:45.770512 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b1c481a-3a33-4bba-af48-f32d9a3cac01 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:45.805590 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8c817-e10f-4515-b7ac-4193e0142a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:45.806385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8c817-e10f-4515-b7ac-4193e0142a05 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:45.806674 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8c817-e10f-4515-b7ac-4193e0142a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:45.806958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b8c817-e10f-4515-b7ac-4193e0142a05 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:45.817913 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b8c817-e10f-4515-b7ac-4193e0142a05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:57:45.825955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-92b8c817-e10f-4515-b7ac-4193e0142a05 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:57:45.825955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:45.825955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:45.825955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:45.825955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:45.825955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:45.825955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:45.825955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:45.825955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:45.825955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:45.825955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:45.825955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:45.825955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:57:45.825955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:57:45.825955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:57:45.825955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:57:45.827054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:57:45.827054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:57:45.827054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:57:45.827054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:57:45.827054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:45.827054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:45.827054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:45.827054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:45.827054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:45.827054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:45.827054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:45.827054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:57:45.827054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:45.827054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:45.827054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:45.827054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:45.828020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:45.828020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:45.828020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:45.828020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:45.828020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:57:45.828020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:57:45.828020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:57:45.828020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:57:45.828020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:57:45.828020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:57:45.828020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:57:45.828020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:57:45.828020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 626/1263] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:57:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:45.833962 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e810f-14aa-434b-91c1-17a3036d11ad None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:45.863071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:45.952921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9e810f-14aa-434b-91c1-17a3036d11ad None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:45.988383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd292da-63a4-42a2-9d79-39161746ea3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:45.989096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd292da-63a4-42a2-9d79-39161746ea3d None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:45.989387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd292da-63a4-42a2-9d79-39161746ea3d None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:45.989673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd292da-63a4-42a2-9d79-39161746ea3d None admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:45.990428 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd292da-63a4-42a2-9d79-39161746ea3d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:45.990742 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd292da-63a4-42a2-9d79-39161746ea3d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:45.992159 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd292da-63a4-42a2-9d79-39161746ea3d None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:46.000417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 638/1264] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:57:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:46.006437 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8c817-e10f-4515-b7ac-4193e0142a05 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:46.077062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:46.112106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b8c817-e10f-4515-b7ac-4193e0142a05 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:46.145314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cfd6aa-5dc6-4944-b9fa-6e3f34c95d16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:46.146116 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cfd6aa-5dc6-4944-b9fa-6e3f34c95d16 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:46.146405 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cfd6aa-5dc6-4944-b9fa-6e3f34c95d16 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:46.146686 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12cfd6aa-5dc6-4944-b9fa-6e3f34c95d16 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:46.157029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cfd6aa-5dc6-4944-b9fa-6e3f34c95d16 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:46.158487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12cfd6aa-5dc6-4944-b9fa-6e3f34c95d16 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:46.175357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-12cfd6aa-5dc6-4944-b9fa-6e3f34c95d16 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:46.175357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:46.175357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:46.175357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:46.175357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:46.175357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:46.175357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:46.175357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:46.175357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:46.175357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:46.175357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:46.175357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:46.175357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:57:46.175357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:57:46.175357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:57:46.176409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:57:46.176409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:46.176409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:46.176409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:46.176409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:46.176409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:46.176409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:46.176409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:46.176409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:57:46.176409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:46.176409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:46.176409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:46.176409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:46.176409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:46.176409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:46.176409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:46.176409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:46.177336 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:57:46.177336 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:57:46.177336 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:46.177336 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:57:46.177336 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 627/1265] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:46.182687 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd292da-63a4-42a2-9d79-39161746ea3d None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:46.267876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:46.295529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd292da-63a4-42a2-9d79-39161746ea3d None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:46.332235 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a984cb-bde1-40dc-8ef0-388dcf2e6140 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:46.332907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a984cb-bde1-40dc-8ef0-388dcf2e6140 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:46.333201 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a984cb-bde1-40dc-8ef0-388dcf2e6140 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:46.333482 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a984cb-bde1-40dc-8ef0-388dcf2e6140 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:46.334289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a984cb-bde1-40dc-8ef0-388dcf2e6140 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:46.334616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a984cb-bde1-40dc-8ef0-388dcf2e6140 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:46.336026 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a984cb-bde1-40dc-8ef0-388dcf2e6140 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:46.348092 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 639/1266] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:46.356615 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cfd6aa-5dc6-4944-b9fa-6e3f34c95d16 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:46.465663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12cfd6aa-5dc6-4944-b9fa-6e3f34c95d16 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:46.486863 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:46.503776 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1e09d-fe59-4135-bc59-983b3701d50e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:46.504644 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1e09d-fe59-4135-bc59-983b3701d50e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:46.504934 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1e09d-fe59-4135-bc59-983b3701d50e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:46.505212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a1e09d-fe59-4135-bc59-983b3701d50e None admin] PATH_INFO: `/v3/users/placement` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:46.513305 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1e09d-fe59-4135-bc59-983b3701d50e None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:46.514753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a1e09d-fe59-4135-bc59-983b3701d50e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:46.529659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-e3a1e09d-fe59-4135-bc59-983b3701d50e None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 15:57:46.529659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:46.529659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:46.529659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:46.529659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:46.529659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:46.529659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:46.529659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:46.529659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:46.529659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:46.529659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:46.529659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:46.529659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:57:46.529659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:57:46.529659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:57:46.529659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:46.531211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:46.532518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:46.532518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:46.532518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:57:46.532518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:57:46.532518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:57:46.532518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:57:46.532518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:57:46.532518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:57:46.532518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 15:57:46.532518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:57:46.532518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 628/1267] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) May 25 15:57:46.538300 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a984cb-bde1-40dc-8ef0-388dcf2e6140 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:46.648116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a984cb-bde1-40dc-8ef0-388dcf2e6140 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:46.683329 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68e110-9191-4f45-92d9-3783202483bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:46.684020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68e110-9191-4f45-92d9-3783202483bd None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:46.684309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68e110-9191-4f45-92d9-3783202483bd None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:46.684578 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68e110-9191-4f45-92d9-3783202483bd None admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:46.685426 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68e110-9191-4f45-92d9-3783202483bd None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:46.685742 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68e110-9191-4f45-92d9-3783202483bd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:46.687181 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68e110-9191-4f45-92d9-3783202483bd None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:46.696231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:46.705148 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 640/1268] 10.222.0.85 () {56 vars in 1143 bytes} [Wed May 25 15:57:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 15:57:46.711264 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1e09d-fe59-4135-bc59-983b3701d50e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:46.823593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a1e09d-fe59-4135-bc59-983b3701d50e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:46.858739 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ca5b7-da84-410c-83c2-58e94f8db86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:46.859529 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ca5b7-da84-410c-83c2-58e94f8db86e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:46.859850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ca5b7-da84-410c-83c2-58e94f8db86e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:46.860133 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3ca5b7-da84-410c-83c2-58e94f8db86e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:46.871426 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ca5b7-da84-410c-83c2-58e94f8db86e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:46.872921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3ca5b7-da84-410c-83c2-58e94f8db86e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:46.898538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-ad3ca5b7-da84-410c-83c2-58e94f8db86e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:46.898538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:46.898538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:46.898538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:46.898538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:46.898538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:46.898538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:46.898538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:46.898538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:46.898538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:46.898538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:46.898538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:46.898538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:57:46.898538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:57:46.898538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:57:46.899609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:57:46.899609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:46.899609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:46.899609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:46.899609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:46.899609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:46.899609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:46.899609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:46.899609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:57:46.899609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:46.899609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:46.899609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:46.899609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:46.899609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:46.899609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:46.899609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:46.899609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:46.905554 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:57:46.905554 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:57:46.905554 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:46.905554 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:57:46.905554 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 629/1269] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:46.906870 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68e110-9191-4f45-92d9-3783202483bd None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:47.021131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68e110-9191-4f45-92d9-3783202483bd None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:47.056894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ca232-7e80-4d64-b9f1-e7aa24e35bc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:47.057679 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ca232-7e80-4d64-b9f1-e7aa24e35bc0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:47.058014 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ca232-7e80-4d64-b9f1-e7aa24e35bc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:47.058301 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9ca232-7e80-4d64-b9f1-e7aa24e35bc0 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:47.059331 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ca232-7e80-4d64-b9f1-e7aa24e35bc0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:47.059692 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ca232-7e80-4d64-b9f1-e7aa24e35bc0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:47.061399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9ca232-7e80-4d64-b9f1-e7aa24e35bc0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:47.074968 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 641/1270] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:57:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:47.081366 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ca5b7-da84-410c-83c2-58e94f8db86e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:47.115614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:47.199533 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3ca5b7-da84-410c-83c2-58e94f8db86e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:47.234094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86229360-f770-433f-9906-747ac5396a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:47.234887 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86229360-f770-433f-9906-747ac5396a6e None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:47.235169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86229360-f770-433f-9906-747ac5396a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:47.235417 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86229360-f770-433f-9906-747ac5396a6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:47.242547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86229360-f770-433f-9906-747ac5396a6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:47.244038 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86229360-f770-433f-9906-747ac5396a6e None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:47.258445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-86229360-f770-433f-9906-747ac5396a6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:57:47.258445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:47.258445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:47.258445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:47.258445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:47.258445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:47.258445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:47.258445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:47.258445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:47.258445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:47.258445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:47.258445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:47.258445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:57:47.258445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:57:47.258445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:57:47.259502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:57:47.259502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:47.259502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:47.259502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:47.259502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:47.259502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:47.259502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:47.259502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:47.259502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:57:47.259502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:47.259502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:47.259502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:47.259502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:47.259502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:47.259502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:47.259502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:47.259502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:47.260726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:57:47.260726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:57:47.260726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:57:47.260726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:57:47.260726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:57:47.260726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:57:47.260726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:57:47.260726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:57:47.260726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 630/1271] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:57:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:47.265671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ca232-7e80-4d64-b9f1-e7aa24e35bc0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:47.332730 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:47.385080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9ca232-7e80-4d64-b9f1-e7aa24e35bc0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:47.421111 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abffa81-6f15-47cf-8369-cda9f77bd791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:47.421776 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abffa81-6f15-47cf-8369-cda9f77bd791 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:47.422022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abffa81-6f15-47cf-8369-cda9f77bd791 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:47.422331 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abffa81-6f15-47cf-8369-cda9f77bd791 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:47.423132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abffa81-6f15-47cf-8369-cda9f77bd791 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:47.423452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abffa81-6f15-47cf-8369-cda9f77bd791 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:47.424948 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abffa81-6f15-47cf-8369-cda9f77bd791 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:47.437152 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 642/1272] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:57:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:47.447202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86229360-f770-433f-9906-747ac5396a6e None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:47.547425 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:47.558107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86229360-f770-433f-9906-747ac5396a6e None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:47.594014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0375bb3e-286c-4b4b-8cba-821e9517bec3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:47.594840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375bb3e-286c-4b4b-8cba-821e9517bec3 None admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:47.595133 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375bb3e-286c-4b4b-8cba-821e9517bec3 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:47.595410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0375bb3e-286c-4b4b-8cba-821e9517bec3 None admin] PATH_INFO: `/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/82b738f1b8c64590955a640c461b4ef9/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:47.597507 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0375bb3e-286c-4b4b-8cba-821e9517bec3 None admin] RBAC: Authorizing `identity:create_grant(project_id=5584ebdaf133444ca69efc5554bf48e9, user_id=82b738f1b8c64590955a640c461b4ef9, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:47.598955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0375bb3e-286c-4b4b-8cba-821e9517bec3 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:47.613977 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 631/1273] 10.222.0.85 () {58 vars in 1439 bytes} [Wed May 25 15:57:47 2022] PUT /identity/v3/projects/5584ebdaf133444ca69efc5554bf48e9/users/82b738f1b8c64590955a640c461b4ef9/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 15:57:49.768346 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5441f-179d-4d38-947c-ea7ae3b72a23 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:49.768929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5441f-179d-4d38-947c-ea7ae3b72a23 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:49.768929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef5441f-179d-4d38-947c-ea7ae3b72a23 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:49.770303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 643/1274] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:57:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:57:49.776946 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8e8e8-57c8-4bff-a2ec-d2bb3fcd27fc None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:49.777691 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8e8e8-57c8-4bff-a2ec-d2bb3fcd27fc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:49.777872 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57c8e8e8-57c8-4bff-a2ec-d2bb3fcd27fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:49.798728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-57c8e8e8-57c8-4bff-a2ec-d2bb3fcd27fc None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:57:49.799951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-57c8e8e8-57c8-4bff-a2ec-d2bb3fcd27fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:57:49.869499 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 632/1275] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:57:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:57:49.897076 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5441f-179d-4d38-947c-ea7ae3b72a23 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:50.023542 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef5441f-179d-4d38-947c-ea7ae3b72a23 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:50.042757 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:50.070752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eac719-8cb5-49c9-9a4b-82354a47dee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:50.071746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eac719-8cb5-49c9-9a4b-82354a47dee1 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:50.072082 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eac719-8cb5-49c9-9a4b-82354a47dee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:50.072421 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72eac719-8cb5-49c9-9a4b-82354a47dee1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:50.085372 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72eac719-8cb5-49c9-9a4b-82354a47dee1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} May 25 15:57:50.094558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-72eac719-8cb5-49c9-9a4b-82354a47dee1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:57:50.094558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:50.094558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:50.094558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:50.094558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:50.094558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:50.094558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:50.094558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:50.094558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:50.094558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:50.094558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:50.094558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:50.094558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get May 25 15:57:50.094558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(role_id) May 25 15:57:50.094558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role May 25 15:57:50.094558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') May 25 15:57:50.095681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call May 25 15:57:50.095681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( May 25 15:57:50.095681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data May 25 15:57:50.095681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) May 25 15:57:50.095681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:50.095681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:50.095681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:50.095681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:50.095681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:50.095681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:50.095681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:50.095681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:57:50.095681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:50.095681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:50.095681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:50.095681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:50.096772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:50.096772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:50.096772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:50.096772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:50.096772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role May 25 15:57:50.096772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) May 25 15:57:50.096772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role May 25 15:57:50.096772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() May 25 15:57:50.096772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role May 25 15:57:50.096772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) May 25 15:57:50.096772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. May 25 15:57:50.096772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:50.096772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 644/1276] 10.222.0.85 () {56 vars in 1096 bytes} [Wed May 25 15:57:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:50.102036 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8e8e8-57c8-4bff-a2ec-d2bb3fcd27fc None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:50.183347 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:50.202985 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57c8e8e8-57c8-4bff-a2ec-d2bb3fcd27fc None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:50.238076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ae885-979d-4442-8c2e-a1572577e605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:50.238749 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ae885-979d-4442-8c2e-a1572577e605 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:50.239040 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ae885-979d-4442-8c2e-a1572577e605 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:50.239323 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011ae885-979d-4442-8c2e-a1572577e605 None admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:50.240171 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ae885-979d-4442-8c2e-a1572577e605 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:50.240490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ae885-979d-4442-8c2e-a1572577e605 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:50.241881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011ae885-979d-4442-8c2e-a1572577e605 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:50.250363 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 633/1277] 10.222.0.85 () {56 vars in 1099 bytes} [Wed May 25 15:57:50 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:50.256841 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eac719-8cb5-49c9-9a4b-82354a47dee1 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:50.369250 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72eac719-8cb5-49c9-9a4b-82354a47dee1 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:50.409700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebcf67f-ce93-4667-8fb9-4adf62fd13e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:50.410901 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcf67f-ce93-4667-8fb9-4adf62fd13e8 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:50.411288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcf67f-ce93-4667-8fb9-4adf62fd13e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:50.411694 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ebcf67f-ce93-4667-8fb9-4adf62fd13e8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:50.429013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebcf67f-ce93-4667-8fb9-4adf62fd13e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:50.430499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ebcf67f-ce93-4667-8fb9-4adf62fd13e8 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:50.448659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-2ebcf67f-ce93-4667-8fb9-4adf62fd13e8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:50.448659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:50.448659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:50.448659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:50.448659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:50.448659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:50.448659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:50.448659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:50.448659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:50.448659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:50.448659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:50.448659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:50.448659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:57:50.448659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:57:50.448659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:57:50.449743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:57:50.449743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:50.449743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:50.449743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:50.449743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:50.449743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:50.449743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:50.449743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:50.449743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:57:50.449743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:50.449743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:50.449743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:50.449743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:50.449743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:50.449743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:50.449743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:50.449743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:50.450652 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:57:50.450652 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:57:50.450652 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:50.450652 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:50.450652 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 645/1278] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:50.458320 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ae885-979d-4442-8c2e-a1572577e605 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:50.564324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011ae885-979d-4442-8c2e-a1572577e605 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:50.599318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd6b78-7754-44df-b162-a85e43b7182b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:50.600009 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd6b78-7754-44df-b162-a85e43b7182b None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:50.600296 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd6b78-7754-44df-b162-a85e43b7182b None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:50.600582 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72fd6b78-7754-44df-b162-a85e43b7182b None admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:50.601303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd6b78-7754-44df-b162-a85e43b7182b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:50.601615 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd6b78-7754-44df-b162-a85e43b7182b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:50.603013 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72fd6b78-7754-44df-b162-a85e43b7182b None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:50.614756 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 634/1279] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:57:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:50.621279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebcf67f-ce93-4667-8fb9-4adf62fd13e8 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:50.658326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:50.731766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ebcf67f-ce93-4667-8fb9-4adf62fd13e8 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:50.766727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da2965-0588-4676-b9fd-7674a693a3ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:50.767523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da2965-0588-4676-b9fd-7674a693a3ef None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:50.767813 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da2965-0588-4676-b9fd-7674a693a3ef None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:50.768127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67da2965-0588-4676-b9fd-7674a693a3ef None admin] PATH_INFO: `/v3/users/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:50.776801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da2965-0588-4676-b9fd-7674a693a3ef None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:50.778274 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67da2965-0588-4676-b9fd-7674a693a3ef None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:50.792986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-67da2965-0588-4676-b9fd-7674a693a3ef None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. May 25 15:57:50.792986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:50.792986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:50.792986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:50.792986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:50.792986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:50.792986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:50.792986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:50.792986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:50.792986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:50.792986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:50.792986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:50.792986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get May 25 15:57:50.792986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_user(user_id) May 25 15:57:50.792986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user May 25 15:57:50.792986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:50.794062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:50.795257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:50.795257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:50.795257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user May 25 15:57:50.795257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) May 25 15:57:50.795257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user May 25 15:57:50.795257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) May 25 15:57:50.795257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user May 25 15:57:50.795257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) May 25 15:57:50.795257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. May 25 15:57:50.795257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:50.795257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 646/1280] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:50.800230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd6b78-7754-44df-b162-a85e43b7182b None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:50.804249 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:50.903262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72fd6b78-7754-44df-b162-a85e43b7182b None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:50.941354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934e181-ec39-4969-927c-c33743244a86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:50.942036 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934e181-ec39-4969-927c-c33743244a86 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:50.942323 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934e181-ec39-4969-927c-c33743244a86 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:50.942654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7934e181-ec39-4969-927c-c33743244a86 None admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:50.943500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934e181-ec39-4969-927c-c33743244a86 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:50.943850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934e181-ec39-4969-927c-c33743244a86 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:50.946298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7934e181-ec39-4969-927c-c33743244a86 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:50.965345 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 635/1281] 10.222.0.85 () {56 vars in 1143 bytes} [Wed May 25 15:57:50 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:50.975119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da2965-0588-4676-b9fd-7674a693a3ef None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:51.076900 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:51.088427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67da2965-0588-4676-b9fd-7674a693a3ef None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:51.124141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62387ed7-29ca-4cc2-9b0c-12976aa9ee63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:51.124787 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62387ed7-29ca-4cc2-9b0c-12976aa9ee63 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:51.125075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62387ed7-29ca-4cc2-9b0c-12976aa9ee63 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:51.125363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62387ed7-29ca-4cc2-9b0c-12976aa9ee63 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:51.201716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62387ed7-29ca-4cc2-9b0c-12976aa9ee63 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:51.203117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62387ed7-29ca-4cc2-9b0c-12976aa9ee63 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:51.294143 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:51.309203 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 647/1282] 10.222.0.85 () {58 vars in 1282 bytes} [Wed May 25 15:57:50 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) May 25 15:57:51.315688 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934e181-ec39-4969-927c-c33743244a86 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:51.380801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:51.433958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7934e181-ec39-4969-927c-c33743244a86 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:51.470106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddad311-1d29-406c-a6d9-37bf02cd2b47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:51.470832 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddad311-1d29-406c-a6d9-37bf02cd2b47 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:51.471118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddad311-1d29-406c-a6d9-37bf02cd2b47 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:51.471399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ddad311-1d29-406c-a6d9-37bf02cd2b47 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:51.482946 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddad311-1d29-406c-a6d9-37bf02cd2b47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:51.484434 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ddad311-1d29-406c-a6d9-37bf02cd2b47 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:51.503100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-1ddad311-1d29-406c-a6d9-37bf02cd2b47 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:51.503100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:51.503100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:51.503100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:51.503100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:51.503100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:51.503100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:51.503100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:51.503100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:51.503100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:51.503100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:51.503100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:51.503100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get May 25 15:57:51.503100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_domain(domain_id) May 25 15:57:51.503100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain May 25 15:57:51.504203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) May 25 15:57:51.504203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:51.504203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:51.504203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:51.504203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:51.504203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:51.504203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:51.504203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:51.504203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:57:51.504203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:51.504203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:51.504203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:51.504203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:51.504203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:51.504203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:51.504203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:51.504203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:51.505340 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain May 25 15:57:51.505340 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) May 25 15:57:51.505340 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. May 25 15:57:51.505340 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:57:51.508010 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 636/1283] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 15:57:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:51.510714 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62387ed7-29ca-4cc2-9b0c-12976aa9ee63 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:51.623432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62387ed7-29ca-4cc2-9b0c-12976aa9ee63 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:51.659228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbe76b-a289-4543-9204-ee6bb6cf6078 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:51.659891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbe76b-a289-4543-9204-ee6bb6cf6078 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:51.660208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbe76b-a289-4543-9204-ee6bb6cf6078 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:51.660494 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30dbe76b-a289-4543-9204-ee6bb6cf6078 None admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:51.661218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbe76b-a289-4543-9204-ee6bb6cf6078 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:51.661557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbe76b-a289-4543-9204-ee6bb6cf6078 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:51.662943 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30dbe76b-a289-4543-9204-ee6bb6cf6078 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:51.677579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 648/1284] 10.222.0.85 () {56 vars in 1109 bytes} [Wed May 25 15:57:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:51.683709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddad311-1d29-406c-a6d9-37bf02cd2b47 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:51.799904 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ddad311-1d29-406c-a6d9-37bf02cd2b47 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:51.813794 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:51.841961 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16051000-377d-4217-a820-8e7804501ccb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:51.842763 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16051000-377d-4217-a820-8e7804501ccb None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:51.843051 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16051000-377d-4217-a820-8e7804501ccb None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:51.843329 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16051000-377d-4217-a820-8e7804501ccb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:51.849920 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16051000-377d-4217-a820-8e7804501ccb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:51.851374 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16051000-377d-4217-a820-8e7804501ccb None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:51.864814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application [None req-16051000-377d-4217-a820-8e7804501ccb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:57:51.864814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:51.864814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:51.864814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:51.864814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:51.864814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:51.864814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:51.864814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:51.864814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:51.864814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:51.864814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:51.864814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:51.864814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get May 25 15:57:51.864814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(project_id) May 25 15:57:51.864814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project May 25 15:57:51.865902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) May 25 15:57:51.865902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:51.865902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:51.865902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:51.865902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:51.865902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:51.865902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:51.865902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:51.865902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application with Lock( May 25 15:57:51.865902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:51.865902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:51.865902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:51.865902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:51.865902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:51.865902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:51.865902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:51.865902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:51.867028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project May 25 15:57:51.867028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) May 25 15:57:51.867028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project May 25 15:57:51.867028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() May 25 15:57:51.867028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project May 25 15:57:51.867028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) May 25 15:57:51.867028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. May 25 15:57:51.867028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: ERROR keystone.server.flask.application May 25 15:57:51.867028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 637/1285] 10.222.0.85 () {56 vars in 1111 bytes} [Wed May 25 15:57:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:51.871984 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbe76b-a289-4543-9204-ee6bb6cf6078 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:51.935291 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:51.997399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30dbe76b-a289-4543-9204-ee6bb6cf6078 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:52.040346 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a071919-183e-47d8-b86c-cba5302da749 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:52.041012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a071919-183e-47d8-b86c-cba5302da749 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:52.041303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a071919-183e-47d8-b86c-cba5302da749 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:52.041587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a071919-183e-47d8-b86c-cba5302da749 None admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:52.042383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a071919-183e-47d8-b86c-cba5302da749 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:52.042703 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a071919-183e-47d8-b86c-cba5302da749 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:52.044178 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a071919-183e-47d8-b86c-cba5302da749 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:52.058316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 649/1286] 10.222.0.85 () {56 vars in 1148 bytes} [Wed May 25 15:57:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:52.064983 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16051000-377d-4217-a820-8e7804501ccb None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:52.168076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16051000-377d-4217-a820-8e7804501ccb None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:52.206122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb1fec8-0133-4238-8b36-eee24c9254fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:52.207331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1fec8-0133-4238-8b36-eee24c9254fc None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:52.207331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1fec8-0133-4238-8b36-eee24c9254fc None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:52.207331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb1fec8-0133-4238-8b36-eee24c9254fc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:52.208213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1fec8-0133-4238-8b36-eee24c9254fc None admin] RBAC: Adding query filter params (role.id=ca9c39f0ab2542618572a3c75e7684a1, scope.project.id=5584ebdaf133444ca69efc5554bf48e9, user.id=82b738f1b8c64590955a640c461b4ef9) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:52.208535 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1fec8-0133-4238-8b36-eee24c9254fc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:52.209966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb1fec8-0133-4238-8b36-eee24c9254fc None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:52.216335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 638/1287] 10.222.0.85 () {56 vars in 1374 bytes} [Wed May 25 15:57:52 2022] GET /identity/v3/role_assignments?user.id=82b738f1b8c64590955a640c461b4ef9&scope.project.id=5584ebdaf133444ca69efc5554bf48e9&role.id=ca9c39f0ab2542618572a3c75e7684a1 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:54.406662 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05d347-288b-4c07-8b2d-f9fc02481ecf None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:54.407200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05d347-288b-4c07-8b2d-f9fc02481ecf None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:54.407200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f05d347-288b-4c07-8b2d-f9fc02481ecf None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:54.408646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 650/1288] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:57:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:57:54.415036 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67940084-1d39-4816-a99b-86c923e4b541 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:54.415769 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67940084-1d39-4816-a99b-86c923e4b541 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:54.415955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67940084-1d39-4816-a99b-86c923e4b541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:54.443841 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-67940084-1d39-4816-a99b-86c923e4b541 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:57:54.445390 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-67940084-1d39-4816-a99b-86c923e4b541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:57:54.475460 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:54.518909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 639/1289] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:57:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:57:54.541497 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05d347-288b-4c07-8b2d-f9fc02481ecf None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:54.656012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f05d347-288b-4c07-8b2d-f9fc02481ecf None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:54.675205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:54.702836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bc79d-c297-40a3-beee-4fd8dea04d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:54.703651 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bc79d-c297-40a3-beee-4fd8dea04d28 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:54.703983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bc79d-c297-40a3-beee-4fd8dea04d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:54.704269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10bc79d-c297-40a3-beee-4fd8dea04d28 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:54.704932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10bc79d-c297-40a3-beee-4fd8dea04d28 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:54.706351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10bc79d-c297-40a3-beee-4fd8dea04d28 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:54.718767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-e10bc79d-c297-40a3-beee-4fd8dea04d28 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 15:57:54.718767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:54.718767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:57:54.718767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:57:54.718767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:57:54.718767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:57:54.718767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:57:54.718767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:57:54.718767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 15:57:54.718767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:54.718767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:57:54.718767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:54.718767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:57:54.718767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:57:54.718767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:57:54.718767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:57:54.718767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:57:54.720009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:57:54.720009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:57:54.720009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:57:54.720009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:57:54.720009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:57:54.720009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:57:54.720009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:57:54.720009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:57:54.720009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:57:54.720009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:57:54.720009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:57:54.720009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:57:54.720009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:57:54.720009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:57:54.720009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:57:54.720009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:57:54.720941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:57:54.720941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:57:54.720941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:57:54.720941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:57:54.720941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:57:54.720941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:57:54.720941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:57:54.720941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:57:54.720941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:57:54.720941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:57:54.720941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:57:54.720941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:57:54.720941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 15:57:54.720941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:57:54.720941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 651/1290] 10.222.0.85 () {56 vars in 1117 bytes} [Wed May 25 15:57:54 2022] GET /identity/v3/services/placement => generated 90 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:57:54.727140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67940084-1d39-4816-a99b-86c923e4b541 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:54.835869 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67940084-1d39-4816-a99b-86c923e4b541 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:54.871614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a61233-a11f-44cb-be79-32f7d219e218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:54.872314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a61233-a11f-44cb-be79-32f7d219e218 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:54.872604 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a61233-a11f-44cb-be79-32f7d219e218 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:54.872884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a61233-a11f-44cb-be79-32f7d219e218 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:54.873599 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a61233-a11f-44cb-be79-32f7d219e218 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:54.873915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a61233-a11f-44cb-be79-32f7d219e218 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:54.875298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a61233-a11f-44cb-be79-32f7d219e218 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:54.880509 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 640/1291] 10.222.0.85 () {56 vars in 1116 bytes} [Wed May 25 15:57:54 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:54.886184 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bc79d-c297-40a3-beee-4fd8dea04d28 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:54.998570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10bc79d-c297-40a3-beee-4fd8dea04d28 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:55.034449 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80c42645-0ff4-401d-b8a3-9a3b7f5d240c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:55.035100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c42645-0ff4-401d-b8a3-9a3b7f5d240c None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:55.035410 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c42645-0ff4-401d-b8a3-9a3b7f5d240c None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:55.035705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80c42645-0ff4-401d-b8a3-9a3b7f5d240c None admin] PATH_INFO: `/v3/services` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:55.036524 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c42645-0ff4-401d-b8a3-9a3b7f5d240c None admin] RBAC: Adding query filter params (type=placement) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:57:55.036853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c42645-0ff4-401d-b8a3-9a3b7f5d240c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:55.038251 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80c42645-0ff4-401d-b8a3-9a3b7f5d240c None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:55.043260 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 652/1292] 10.222.0.85 () {56 vars in 1116 bytes} [Wed May 25 15:57:54 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:57:57.257918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598be75-5dd1-455a-bb28-9fae3b7e0bbe None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:57.258586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598be75-5dd1-455a-bb28-9fae3b7e0bbe None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:57.258683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1598be75-5dd1-455a-bb28-9fae3b7e0bbe None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:57.260945 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 641/1293] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:57:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:57:57.267356 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab34bca-31a4-4217-9b46-930fc3906651 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:57.268044 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab34bca-31a4-4217-9b46-930fc3906651 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:57.268044 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ab34bca-31a4-4217-9b46-930fc3906651 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:57.290383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0ab34bca-31a4-4217-9b46-930fc3906651 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:57:57.291684 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0ab34bca-31a4-4217-9b46-930fc3906651 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:57:57.323862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:57.362583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 653/1294] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:57:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:57:57.385230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598be75-5dd1-455a-bb28-9fae3b7e0bbe None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:57.428594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:57:57.504857 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1598be75-5dd1-455a-bb28-9fae3b7e0bbe None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:57:57.545119 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73196d9-f769-4d69-b37c-6fefe57325d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:57:57.545803 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73196d9-f769-4d69-b37c-6fefe57325d6 None admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:57.546101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73196d9-f769-4d69-b37c-6fefe57325d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:57.546388 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73196d9-f769-4d69-b37c-6fefe57325d6 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:57.547134 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73196d9-f769-4d69-b37c-6fefe57325d6 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:57:57.548555 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73196d9-f769-4d69-b37c-6fefe57325d6 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:57:57.564423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 642/1295] 10.222.0.85 () {60 vars in 1141 bytes} [Wed May 25 15:57:57 2022] POST /identity/v3/services => generated 246 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 15:57:59.755377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc81ae3-caf7-480a-95b0-d43344f8f85b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:59.756296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc81ae3-caf7-480a-95b0-d43344f8f85b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:59.756296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc81ae3-caf7-480a-95b0-d43344f8f85b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:59.758561 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 654/1296] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:57:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:57:59.764033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d01e8b-b596-4357-b3f7-f299a641ecb4 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:57:59.764473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d01e8b-b596-4357-b3f7-f299a641ecb4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:57:59.764574 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d01e8b-b596-4357-b3f7-f299a641ecb4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:57:59.783140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b1d01e8b-b596-4357-b3f7-f299a641ecb4 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:57:59.784363 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b1d01e8b-b596-4357-b3f7-f299a641ecb4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:57:59.860223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 643/1297] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:57:59 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:57:59.883416 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc81ae3-caf7-480a-95b0-d43344f8f85b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:57:59.994225 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc81ae3-caf7-480a-95b0-d43344f8f85b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:58:00.029204 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823560ae-6b1d-46e0-9baa-1d2e0922f9a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:58:00.029932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823560ae-6b1d-46e0-9baa-1d2e0922f9a0 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:58:00.030225 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823560ae-6b1d-46e0-9baa-1d2e0922f9a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:58:00.030512 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823560ae-6b1d-46e0-9baa-1d2e0922f9a0 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:58:00.031155 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823560ae-6b1d-46e0-9baa-1d2e0922f9a0 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:58:00.032582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823560ae-6b1d-46e0-9baa-1d2e0922f9a0 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:58:00.044551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-823560ae-6b1d-46e0-9baa-1d2e0922f9a0 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 15:58:00.044551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:58:00.044551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:58:00.044551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:58:00.044551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:58:00.044551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:58:00.044551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:58:00.044551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:58:00.044551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 15:58:00.044551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:58:00.044551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:58:00.044551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:58:00.044551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:58:00.044551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:58:00.044551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:58:00.044551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:58:00.044551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:58:00.045669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:58:00.045669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:58:00.045669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:58:00.045669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:58:00.045669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:58:00.045669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:58:00.045669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:58:00.045669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:58:00.045669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:58:00.045669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:58:00.045669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:58:00.045669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:58:00.045669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:58:00.045669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:58:00.045669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:58:00.045669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:58:00.046576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:58:00.046576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:58:00.046576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:58:00.046576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:58:00.046576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:58:00.046576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:58:00.046576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:58:00.046576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:58:00.046576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:58:00.046576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:58:00.046576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:58:00.046576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:58:00.046576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 15:58:00.046576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:58:00.046576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 655/1298] 10.222.0.85 () {56 vars in 1117 bytes} [Wed May 25 15:57:59 2022] GET /identity/v3/services/placement => generated 90 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:58:00.054140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d01e8b-b596-4357-b3f7-f299a641ecb4 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:58:00.083709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:58:00.177408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d01e8b-b596-4357-b3f7-f299a641ecb4 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:58:00.212786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e400f-4940-40f6-b571-33ec4ea4d638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:58:00.213477 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e400f-4940-40f6-b571-33ec4ea4d638 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:58:00.213768 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e400f-4940-40f6-b571-33ec4ea4d638 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:58:00.214047 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20e400f-4940-40f6-b571-33ec4ea4d638 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:58:00.214749 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e400f-4940-40f6-b571-33ec4ea4d638 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:58:00.215061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e400f-4940-40f6-b571-33ec4ea4d638 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:58:00.216464 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20e400f-4940-40f6-b571-33ec4ea4d638 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:58:00.221672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 644/1299] 10.222.0.85 () {56 vars in 1116 bytes} [Wed May 25 15:58:00 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:58:00.228568 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823560ae-6b1d-46e0-9baa-1d2e0922f9a0 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:58:00.265369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:58:00.338879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823560ae-6b1d-46e0-9baa-1d2e0922f9a0 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:58:00.374703 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268ddf00-78de-4f48-bb13-ba5ea1bd9035 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:58:00.375361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ddf00-78de-4f48-bb13-ba5ea1bd9035 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:58:00.375647 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ddf00-78de-4f48-bb13-ba5ea1bd9035 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:58:00.375948 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268ddf00-78de-4f48-bb13-ba5ea1bd9035 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:58:00.376739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ddf00-78de-4f48-bb13-ba5ea1bd9035 None admin] RBAC: Adding query filter params (interface=public, service_id=6432f039551344b080a1aa977f4a5837, region_id=RegionOne) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:58:00.377052 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ddf00-78de-4f48-bb13-ba5ea1bd9035 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:58:00.378434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268ddf00-78de-4f48-bb13-ba5ea1bd9035 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:58:00.383635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 656/1300] 10.222.0.85 () {56 vars in 1251 bytes} [Wed May 25 15:58:00 2022] GET /identity/v3/endpoints?service_id=6432f039551344b080a1aa977f4a5837&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:58:00.389545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e400f-4940-40f6-b571-33ec4ea4d638 None admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:58:00.494742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:58:00.514928 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20e400f-4940-40f6-b571-33ec4ea4d638 None admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:58:00.548777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12f0e37-8059-468c-af06-c5142538bb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:58:00.549443 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f0e37-8059-468c-af06-c5142538bb82 None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:58:00.549740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f0e37-8059-468c-af06-c5142538bb82 None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:58:00.550023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12f0e37-8059-468c-af06-c5142538bb82 None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:58:00.550711 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f0e37-8059-468c-af06-c5142538bb82 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:58:00.552142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12f0e37-8059-468c-af06-c5142538bb82 None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:58:00.558899 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 645/1301] 10.222.0.85 () {56 vars in 1087 bytes} [Wed May 25 15:58:00 2022] GET /identity/v3/services => generated 2191 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 15:58:02.729960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1bff0-bfb8-408f-b2f3-6c3fa13a7ca4 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:58:02.730546 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1bff0-bfb8-408f-b2f3-6c3fa13a7ca4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:58:02.730546 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f1bff0-bfb8-408f-b2f3-6c3fa13a7ca4 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:58:02.731932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 657/1302] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:58:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:58:02.738380 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5ce3e-4f24-44ca-ae9a-0497d9ba8657 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:58:02.739081 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5ce3e-4f24-44ca-ae9a-0497d9ba8657 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:58:02.739254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca5ce3e-4f24-44ca-ae9a-0497d9ba8657 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:58:02.763058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8ca5ce3e-4f24-44ca-ae9a-0497d9ba8657 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:58:02.764322 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8ca5ce3e-4f24-44ca-ae9a-0497d9ba8657 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:58:02.835292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 646/1303] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:58:02 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) May 25 15:58:02.863305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1bff0-bfb8-408f-b2f3-6c3fa13a7ca4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:58:02.963930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:58:02.972565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f1bff0-bfb8-408f-b2f3-6c3fa13a7ca4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:58:03.009499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47481042-f38a-42c2-ab89-7f982f6bd685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:58:03.010232 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47481042-f38a-42c2-ab89-7f982f6bd685 None admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:58:03.010528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47481042-f38a-42c2-ab89-7f982f6bd685 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:58:03.010805 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47481042-f38a-42c2-ab89-7f982f6bd685 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:58:03.011463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47481042-f38a-42c2-ab89-7f982f6bd685 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:58:03.012896 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47481042-f38a-42c2-ab89-7f982f6bd685 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:58:03.024974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application [None req-47481042-f38a-42c2-ab89-7f982f6bd685 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. May 25 15:58:03.024974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:58:03.024974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service May 25 15:58:03.024974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) May 25 15:58:03.024974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service May 25 15:58:03.024974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() May 25 15:58:03.024974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service May 25 15:58:03.024974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:58:03.024974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 15:58:03.024974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:58:03.024974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: May 25 15:58:03.024974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:58:03.024974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application Traceback (most recent call last): May 25 15:58:03.024974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request May 25 15:58:03.024974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application rv = self.dispatch_request() May 25 15:58:03.024974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request May 25 15:58:03.024974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) May 25 15:58:03.026087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper May 25 15:58:03.026087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) May 25 15:58:03.026087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view May 25 15:58:03.026087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) May 25 15:58:03.026087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request May 25 15:58:03.026087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) May 25 15:58:03.026087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get May 25 15:58:03.026087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._get_service(service_id) May 25 15:58:03.026087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service May 25 15:58:03.026087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) May 25 15:58:03.026087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped May 25 15:58:03.026087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) May 25 15:58:03.026087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun May 25 15:58:03.026087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) May 25 15:58:03.026087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func May 25 15:58:03.026087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.get_or_create( May 25 15:58:03.026985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create May 25 15:58:03.026985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application with Lock( May 25 15:58:03.026985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ May 25 15:58:03.026985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self._enter() May 25 15:58:03.026985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter May 25 15:58:03.026985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) May 25 15:58:03.026985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create May 25 15:58:03.026985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application return self.creator() May 25 15:58:03.026985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value May 25 15:58:03.026985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application created_value = creator( May 25 15:58:03.026985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service May 25 15:58:03.026985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) May 25 15:58:03.026985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. May 25 15:58:03.026985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: ERROR keystone.server.flask.application May 25 15:58:03.031341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 658/1304] 10.222.0.85 () {56 vars in 1117 bytes} [Wed May 25 15:58:02 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) May 25 15:58:03.033100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5ce3e-4f24-44ca-ae9a-0497d9ba8657 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:58:03.134052 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:58:03.158182 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca5ce3e-4f24-44ca-ae9a-0497d9ba8657 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:58:03.193016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed0ef4d4-2bd8-436b-838c-dcfe851fa0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:58:03.193681 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ef4d4-2bd8-436b-838c-dcfe851fa0ca None admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:58:03.193973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ef4d4-2bd8-436b-838c-dcfe851fa0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:58:03.194565 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed0ef4d4-2bd8-436b-838c-dcfe851fa0ca None admin] PATH_INFO: `/v3/services` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:58:03.195308 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ef4d4-2bd8-436b-838c-dcfe851fa0ca None admin] RBAC: Adding query filter params (name=placement) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 15:58:03.195626 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ef4d4-2bd8-436b-838c-dcfe851fa0ca None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:58:03.197050 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed0ef4d4-2bd8-436b-838c-dcfe851fa0ca None admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:58:03.202518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 647/1305] 10.222.0.85 () {56 vars in 1116 bytes} [Wed May 25 15:58:03 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 15:58:03.208663 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47481042-f38a-42c2-ab89-7f982f6bd685 None admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:58:03.319070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47481042-f38a-42c2-ab89-7f982f6bd685 None admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:58:03.354384 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f08c520-9ecb-494a-8866-612f3fbe1600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:58:03.355037 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08c520-9ecb-494a-8866-612f3fbe1600 None admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:58:03.355325 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08c520-9ecb-494a-8866-612f3fbe1600 None admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:58:03.355606 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f08c520-9ecb-494a-8866-612f3fbe1600 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:58:03.356397 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08c520-9ecb-494a-8866-612f3fbe1600 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:58:03.357788 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f08c520-9ecb-494a-8866-612f3fbe1600 None admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:58:03.381741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 659/1306] 10.222.0.85 () {60 vars in 1144 bytes} [Wed May 25 15:58:03 2022] POST /identity/v3/endpoints => generated 331 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 15:58:13.199627 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=30 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:58:13.375686 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:06.705654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11901eb2-c8a4-4a55-ab60-72658a39ff88 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:06.706286 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11901eb2-c8a4-4a55-ab60-72658a39ff88 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:06.706286 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11901eb2-c8a4-4a55-ab60-72658a39ff88 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:06.711480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 648/1307] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:59:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:06.720074 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7d39f-e8c1-4839-b358-43c1e4a5a7e2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:06.720622 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7d39f-e8c1-4839-b358-43c1e4a5a7e2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:06.720859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7d39f-e8c1-4839-b358-43c1e4a5a7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:06.865413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d3f7d39f-e8c1-4839-b358-43c1e4a5a7e2 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:06.866519 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d3f7d39f-e8c1-4839-b358-43c1e4a5a7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:07.031856 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 660/1308] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:59:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:59:07.076953 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3497ab0-f63f-47ec-b185-d82ca8d2bf86 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:07.077224 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3497ab0-f63f-47ec-b185-d82ca8d2bf86 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:07.077774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3497ab0-f63f-47ec-b185-d82ca8d2bf86 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:07.079262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 649/1309] 10.222.0.85 () {54 vars in 964 bytes} [Wed May 25 15:59:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:07.085459 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40859b-f881-478b-b5fa-97806a099863 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:07.085769 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40859b-f881-478b-b5fa-97806a099863 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:07.086167 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c40859b-f881-478b-b5fa-97806a099863 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:07.173951 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4c40859b-f881-478b-b5fa-97806a099863 None None] MFA Rules not processed for user `413b3d65427b41dc99bc7b0a94fb1dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:07.175282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4c40859b-f881-478b-b5fa-97806a099863 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:07.197660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:07.256453 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 661/1310] 10.222.0.85 () {58 vars in 1061 bytes} [Wed May 25 15:59:07 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:59:07.265425 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3497ab0-f63f-47ec-b185-d82ca8d2bf86 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:59:07.267545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e3497ab0-f63f-47ec-b185-d82ca8d2bf86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:07.387163 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3497ab0-f63f-47ec-b185-d82ca8d2bf86 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:59:07.467868 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7487ba60-d44e-4278-a8d9-21d93442033c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:59:07.468818 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487ba60-d44e-4278-a8d9-21d93442033c service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:07.469166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487ba60-d44e-4278-a8d9-21d93442033c service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:07.469601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7487ba60-d44e-4278-a8d9-21d93442033c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:07.471334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7487ba60-d44e-4278-a8d9-21d93442033c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:07.529703 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:07.587997 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487ba60-d44e-4278-a8d9-21d93442033c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:59:07.589549 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7487ba60-d44e-4278-a8d9-21d93442033c service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:59:07.609058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 650/1311] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 15:59:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:59:10.078352 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01014a10-bc10-4b09-8ee7-5ce6bf34f10d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:10.078993 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01014a10-bc10-4b09-8ee7-5ce6bf34f10d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:10.079095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01014a10-bc10-4b09-8ee7-5ce6bf34f10d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:10.080562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 662/1312] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:59:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:10.086463 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d597370-1e53-453c-875c-9a789991cdac None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:10.086865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d597370-1e53-453c-875c-9a789991cdac None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:10.087328 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d597370-1e53-453c-875c-9a789991cdac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:10.177989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8d597370-1e53-453c-875c-9a789991cdac None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:10.179333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8d597370-1e53-453c-875c-9a789991cdac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:10.289365 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 651/1313] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:59:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:59:10.341283 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f58da-beb9-4f90-89df-57c45c79654c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:10.341625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f58da-beb9-4f90-89df-57c45c79654c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:10.341862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301f58da-beb9-4f90-89df-57c45c79654c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:10.343372 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 663/1314] 10.222.0.85 () {54 vars in 964 bytes} [Wed May 25 15:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:10.349256 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b689c-943b-4740-a9fd-6155d78552f7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:10.349845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b689c-943b-4740-a9fd-6155d78552f7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:10.350203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b689c-943b-4740-a9fd-6155d78552f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:10.361140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:10.424591 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-968b689c-943b-4740-a9fd-6155d78552f7 None None] MFA Rules not processed for user `413b3d65427b41dc99bc7b0a94fb1dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:10.425823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-968b689c-943b-4740-a9fd-6155d78552f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:10.503547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 652/1315] 10.222.0.85 () {58 vars in 1061 bytes} [Wed May 25 15:59:10 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:59:10.512760 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f58da-beb9-4f90-89df-57c45c79654c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:59:10.514283 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-301f58da-beb9-4f90-89df-57c45c79654c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:10.627769 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301f58da-beb9-4f90-89df-57c45c79654c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:59:10.702953 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:10.705747 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd5cc73-f623-4036-82ca-b4ffcd7ac5e4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:59:10.706388 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5cc73-f623-4036-82ca-b4ffcd7ac5e4 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:10.706779 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5cc73-f623-4036-82ca-b4ffcd7ac5e4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:10.706993 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd5cc73-f623-4036-82ca-b4ffcd7ac5e4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:10.708943 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1dd5cc73-f623-4036-82ca-b4ffcd7ac5e4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:10.827161 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5cc73-f623-4036-82ca-b4ffcd7ac5e4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:59:10.829147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd5cc73-f623-4036-82ca-b4ffcd7ac5e4 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:59:10.849928 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 664/1316] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 15:59:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:59:13.572746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18bb86-d4d9-45b3-b4a9-e79229d97462 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:13.573297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18bb86-d4d9-45b3-b4a9-e79229d97462 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:13.573297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b18bb86-d4d9-45b3-b4a9-e79229d97462 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:13.575330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 653/1317] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:59:13 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:13.581831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84aaa2-1017-44ca-9f97-bc37a1ab2412 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:13.582302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84aaa2-1017-44ca-9f97-bc37a1ab2412 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:13.582400 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b84aaa2-1017-44ca-9f97-bc37a1ab2412 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:13.650082 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8b84aaa2-1017-44ca-9f97-bc37a1ab2412 None None] MFA Rules not processed for user `b9df4b9d4b0942339277d976457b2124`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:13.651286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8b84aaa2-1017-44ca-9f97-bc37a1ab2412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:13.709356 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:13.758028 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 665/1318] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:59:13 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 15:59:13.796840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18bb86-d4d9-45b3-b4a9-e79229d97462 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:59:13.798381 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4b18bb86-d4d9-45b3-b4a9-e79229d97462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:13.906655 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b18bb86-d4d9-45b3-b4a9-e79229d97462 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:59:13.909777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:13.998109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fb542c0-f513-4bf5-a811-cab01dc180bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:59:13.998765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb542c0-f513-4bf5-a811-cab01dc180bf service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:13.999051 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb542c0-f513-4bf5-a811-cab01dc180bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:13.999333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb542c0-f513-4bf5-a811-cab01dc180bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:14.001224 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7fb542c0-f513-4bf5-a811-cab01dc180bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:14.115121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb542c0-f513-4bf5-a811-cab01dc180bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:59:14.116589 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fb542c0-f513-4bf5-a811-cab01dc180bf service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:59:14.135356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 654/1319] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 15:59:13 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:59:16.868745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11b347-6a9e-4f21-b569-c64d6b0e06ea None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:16.869334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11b347-6a9e-4f21-b569-c64d6b0e06ea None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:16.869334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee11b347-6a9e-4f21-b569-c64d6b0e06ea None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:16.870677 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 666/1320] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:16.877327 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91d643-8cdd-4b08-b99c-f4ea98fa8f5c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:16.878063 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91d643-8cdd-4b08-b99c-f4ea98fa8f5c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:16.878214 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a91d643-8cdd-4b08-b99c-f4ea98fa8f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:16.930449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:16.958032 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-9a91d643-8cdd-4b08-b99c-f4ea98fa8f5c None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:16.959261 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9a91d643-8cdd-4b08-b99c-f4ea98fa8f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:17.068391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 655/1321] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:59:16 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:59:17.120404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11b347-6a9e-4f21-b569-c64d6b0e06ea None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:59:17.121961 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ee11b347-6a9e-4f21-b569-c64d6b0e06ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:17.232437 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee11b347-6a9e-4f21-b569-c64d6b0e06ea None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:59:17.310031 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f011c9-bf35-4b49-9c80-5db4ed496ce4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:59:17.310694 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f011c9-bf35-4b49-9c80-5db4ed496ce4 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:17.310989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f011c9-bf35-4b49-9c80-5db4ed496ce4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:17.311277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f011c9-bf35-4b49-9c80-5db4ed496ce4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:17.313012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a1f011c9-bf35-4b49-9c80-5db4ed496ce4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:17.321997 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:17.427743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f011c9-bf35-4b49-9c80-5db4ed496ce4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:59:17.429239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1f011c9-bf35-4b49-9c80-5db4ed496ce4 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:59:17.449086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 667/1322] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 15:59:17 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:59:20.499541 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de58ec-b0f2-4b66-bb8a-f370d196e32d None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:20.500229 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de58ec-b0f2-4b66-bb8a-f370d196e32d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:20.500229 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19de58ec-b0f2-4b66-bb8a-f370d196e32d None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:20.501546 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 656/1323] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:20.507201 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d87f71-0c82-46b2-bfb1-1519297e346c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:20.507668 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d87f71-0c82-46b2-bfb1-1519297e346c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:20.507935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d87f71-0c82-46b2-bfb1-1519297e346c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:20.576499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c1d87f71-0c82-46b2-bfb1-1519297e346c None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:20.577665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c1d87f71-0c82-46b2-bfb1-1519297e346c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:20.638115 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:20.684450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 668/1324] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:59:20.721006 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de58ec-b0f2-4b66-bb8a-f370d196e32d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:59:20.722573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-19de58ec-b0f2-4b66-bb8a-f370d196e32d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:20.831200 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19de58ec-b0f2-4b66-bb8a-f370d196e32d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:59:20.844714 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:20.909520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a683484-e99c-46c4-aa76-39deb8d7dcce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:59:20.910168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a683484-e99c-46c4-aa76-39deb8d7dcce service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:20.910456 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a683484-e99c-46c4-aa76-39deb8d7dcce service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:20.910738 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a683484-e99c-46c4-aa76-39deb8d7dcce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:20.912432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7a683484-e99c-46c4-aa76-39deb8d7dcce service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:21.026145 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a683484-e99c-46c4-aa76-39deb8d7dcce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:59:21.027603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a683484-e99c-46c4-aa76-39deb8d7dcce service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:59:21.046915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 657/1325] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 15:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:59:23.300869 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26400db5-0267-458d-adea-d458d8816baf None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:23.301402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26400db5-0267-458d-adea-d458d8816baf None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:23.301402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26400db5-0267-458d-adea-d458d8816baf None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:23.303236 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 669/1326] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:23.309018 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0741708-8877-4c27-8d64-cfe11284d5e1 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:23.309433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0741708-8877-4c27-8d64-cfe11284d5e1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:23.309714 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0741708-8877-4c27-8d64-cfe11284d5e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:23.357538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:23.376549 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-a0741708-8877-4c27-8d64-cfe11284d5e1 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:23.377798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a0741708-8877-4c27-8d64-cfe11284d5e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:23.477111 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 658/1327] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:59:23.512303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26400db5-0267-458d-adea-d458d8816baf None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:59:23.513844 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-26400db5-0267-458d-adea-d458d8816baf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:23.619053 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26400db5-0267-458d-adea-d458d8816baf None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:59:23.695212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb20e09f-8b4c-4f3a-b873-51f81b67edc3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:59:23.695875 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20e09f-8b4c-4f3a-b873-51f81b67edc3 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:23.696175 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20e09f-8b4c-4f3a-b873-51f81b67edc3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:23.696479 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb20e09f-8b4c-4f3a-b873-51f81b67edc3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:23.698119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bb20e09f-8b4c-4f3a-b873-51f81b67edc3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:23.715544 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:23.816806 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20e09f-8b4c-4f3a-b873-51f81b67edc3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:59:23.818290 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb20e09f-8b4c-4f3a-b873-51f81b67edc3 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:59:23.837405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 670/1328] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 15:59:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:59:26.089493 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da92fc2-4444-4abc-ba4f-518b22f8f92f None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:26.090099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da92fc2-4444-4abc-ba4f-518b22f8f92f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:26.090099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da92fc2-4444-4abc-ba4f-518b22f8f92f None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:26.091449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 659/1329] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:59:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:26.096950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20113e93-0eac-44d7-9149-ac45a62ac6e6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:26.097408 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20113e93-0eac-44d7-9149-ac45a62ac6e6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:26.097541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20113e93-0eac-44d7-9149-ac45a62ac6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:26.165596 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-20113e93-0eac-44d7-9149-ac45a62ac6e6 None None] MFA Rules not processed for user `b9df4b9d4b0942339277d976457b2124`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:26.166881 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-20113e93-0eac-44d7-9149-ac45a62ac6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:26.234792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:26.270336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 671/1330] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:59:26 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:59:26.307473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da92fc2-4444-4abc-ba4f-518b22f8f92f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:59:26.309101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4da92fc2-4444-4abc-ba4f-518b22f8f92f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:26.415271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da92fc2-4444-4abc-ba4f-518b22f8f92f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:59:26.436990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:26.492477 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96513a38-3856-44e5-893f-a0812daf2f47 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:59:26.493138 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96513a38-3856-44e5-893f-a0812daf2f47 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:26.493430 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96513a38-3856-44e5-893f-a0812daf2f47 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:26.493713 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96513a38-3856-44e5-893f-a0812daf2f47 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:26.495349 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-96513a38-3856-44e5-893f-a0812daf2f47 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:26.606887 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96513a38-3856-44e5-893f-a0812daf2f47 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:59:26.608383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96513a38-3856-44e5-893f-a0812daf2f47 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:59:26.627872 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 660/1331] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 15:59:26 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:59:29.099850 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566344b-cf23-4d93-9ff2-0168e6688bcb None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:29.100350 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566344b-cf23-4d93-9ff2-0168e6688bcb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:29.100350 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566344b-cf23-4d93-9ff2-0168e6688bcb None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:29.101736 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 672/1332] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:59:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:29.107259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef216f59-e298-4a0b-b637-572727e66cf8 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:29.107733 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef216f59-e298-4a0b-b637-572727e66cf8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:29.107867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef216f59-e298-4a0b-b637-572727e66cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:29.172085 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:29.176941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ef216f59-e298-4a0b-b637-572727e66cf8 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:29.179009 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ef216f59-e298-4a0b-b637-572727e66cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:29.285164 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 661/1333] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:59:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:59:29.320671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566344b-cf23-4d93-9ff2-0168e6688bcb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:59:29.322227 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6566344b-cf23-4d93-9ff2-0168e6688bcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:29.433949 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6566344b-cf23-4d93-9ff2-0168e6688bcb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:59:29.510026 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ca29726-14f9-470e-9b40-fcbc6f7b493f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:59:29.510691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca29726-14f9-470e-9b40-fcbc6f7b493f service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:29.510982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca29726-14f9-470e-9b40-fcbc6f7b493f service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:29.511262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ca29726-14f9-470e-9b40-fcbc6f7b493f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:29.512972 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8ca29726-14f9-470e-9b40-fcbc6f7b493f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:29.542234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:29.627483 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca29726-14f9-470e-9b40-fcbc6f7b493f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:59:29.628968 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ca29726-14f9-470e-9b40-fcbc6f7b493f service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:59:29.647644 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 673/1334] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 15:59:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:59:31.889287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75f50a-1481-4d98-8bfc-dbc43b67e5c8 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:31.890208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75f50a-1481-4d98-8bfc-dbc43b67e5c8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:31.890208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b75f50a-1481-4d98-8bfc-dbc43b67e5c8 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:31.892554 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 662/1335] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:59:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:31.898913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad23dc-c865-4b6b-a7a4-2876d5385741 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:31.899645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad23dc-c865-4b6b-a7a4-2876d5385741 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:31.899796 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecad23dc-c865-4b6b-a7a4-2876d5385741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:31.971303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ecad23dc-c865-4b6b-a7a4-2876d5385741 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:31.972566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ecad23dc-c865-4b6b-a7a4-2876d5385741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:32.055029 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:32.080048 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 674/1336] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:59:32.119330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75f50a-1481-4d98-8bfc-dbc43b67e5c8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:59:32.120885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3b75f50a-1481-4d98-8bfc-dbc43b67e5c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:32.232579 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b75f50a-1481-4d98-8bfc-dbc43b67e5c8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:59:32.262123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:32.308109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62687fe2-179f-452e-864d-ec4f8e7b97fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:59:32.308871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62687fe2-179f-452e-864d-ec4f8e7b97fc service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:32.309346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62687fe2-179f-452e-864d-ec4f8e7b97fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:32.309692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62687fe2-179f-452e-864d-ec4f8e7b97fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:32.312650 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-62687fe2-179f-452e-864d-ec4f8e7b97fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:32.450621 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62687fe2-179f-452e-864d-ec4f8e7b97fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:59:32.452502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62687fe2-179f-452e-864d-ec4f8e7b97fc service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:59:32.472692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 663/1337] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 15:59:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:59:35.031567 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e9b82-3f28-4cea-8dbe-a0b3fde6df67 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:35.032608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e9b82-3f28-4cea-8dbe-a0b3fde6df67 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:35.032608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845e9b82-3f28-4cea-8dbe-a0b3fde6df67 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:35.034641 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 675/1338] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:59:35 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:35.041178 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce1918-78aa-416f-b1cb-eeaed8435fad None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:35.041881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce1918-78aa-416f-b1cb-eeaed8435fad None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:35.042078 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce1918-78aa-416f-b1cb-eeaed8435fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:35.117220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-abce1918-78aa-416f-b1cb-eeaed8435fad None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:35.118463 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-abce1918-78aa-416f-b1cb-eeaed8435fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:35.126933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:35.218965 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 664/1339] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:59:35 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:59:35.257302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e9b82-3f28-4cea-8dbe-a0b3fde6df67 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:59:35.258856 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-845e9b82-3f28-4cea-8dbe-a0b3fde6df67 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:35.367318 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845e9b82-3f28-4cea-8dbe-a0b3fde6df67 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:59:35.444520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6c52d0-a3ff-4251-b37a-abba7ee3cfdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:59:35.445177 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c52d0-a3ff-4251-b37a-abba7ee3cfdc service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:35.445466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c52d0-a3ff-4251-b37a-abba7ee3cfdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:35.445748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6c52d0-a3ff-4251-b37a-abba7ee3cfdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:35.447422 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-db6c52d0-a3ff-4251-b37a-abba7ee3cfdc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:35.495582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:35.571242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c52d0-a3ff-4251-b37a-abba7ee3cfdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:59:35.572660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6c52d0-a3ff-4251-b37a-abba7ee3cfdc service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:59:35.590941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 676/1340] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 15:59:35 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 15:59:40.001725 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7850b-04cb-41de-b23e-012b0167c1b6 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:40.002418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7850b-04cb-41de-b23e-012b0167c1b6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:40.002418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7c7850b-04cb-41de-b23e-012b0167c1b6 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:40.003595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 665/1341] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:40.009110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33106d1c-e39a-408a-88c3-a325047ffa45 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:40.009545 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33106d1c-e39a-408a-88c3-a325047ffa45 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:40.009719 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33106d1c-e39a-408a-88c3-a325047ffa45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:40.092723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-33106d1c-e39a-408a-88c3-a325047ffa45 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:40.093981 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-33106d1c-e39a-408a-88c3-a325047ffa45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:40.196981 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:40.211426 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 677/1342] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:59:40.256319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7850b-04cb-41de-b23e-012b0167c1b6 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:59:40.258697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f7c7850b-04cb-41de-b23e-012b0167c1b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:40.361202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7c7850b-04cb-41de-b23e-012b0167c1b6 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:59:40.364917 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbc4d2b-c17b-4150-a0f5-9c02d3167b2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:59:40.365582 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc4d2b-c17b-4150-a0f5-9c02d3167b2d service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:40.365873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc4d2b-c17b-4150-a0f5-9c02d3167b2d service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:40.366164 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbc4d2b-c17b-4150-a0f5-9c02d3167b2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:40.375254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc4d2b-c17b-4150-a0f5-9c02d3167b2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:59:40.376727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbc4d2b-c17b-4150-a0f5-9c02d3167b2d service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:59:40.386048 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 666/1343] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 15:59:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:59:43.144180 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0a1b0-aff3-4606-a7d4-0cee56469869 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:43.144762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0a1b0-aff3-4606-a7d4-0cee56469869 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:43.144762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d0a1b0-aff3-4606-a7d4-0cee56469869 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:43.146105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 678/1344] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:43.151445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833bce7-78fd-4d59-ba30-a1bdfff3b47e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:43.151918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833bce7-78fd-4d59-ba30-a1bdfff3b47e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:43.152017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a833bce7-78fd-4d59-ba30-a1bdfff3b47e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:43.181247 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-a833bce7-78fd-4d59-ba30-a1bdfff3b47e None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:43.182491 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a833bce7-78fd-4d59-ba30-a1bdfff3b47e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:43.212292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:43.264429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 667/1345] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:59:43 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:59:43.301219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0a1b0-aff3-4606-a7d4-0cee56469869 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:59:43.314524 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d0a1b0-aff3-4606-a7d4-0cee56469869 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:59:43.319027 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96042ae9-12dd-43f8-a014-9f67e19cdf7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:59:43.320158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96042ae9-12dd-43f8-a014-9f67e19cdf7e service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:43.320671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96042ae9-12dd-43f8-a014-9f67e19cdf7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:43.321191 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96042ae9-12dd-43f8-a014-9f67e19cdf7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:43.331834 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96042ae9-12dd-43f8-a014-9f67e19cdf7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:59:43.333371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96042ae9-12dd-43f8-a014-9f67e19cdf7e service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:59:43.343574 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 679/1346] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 15:59:43 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:59:47.543573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefeff5-7ca6-4ca7-b09d-0204f4d7fa3d None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:47.544156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefeff5-7ca6-4ca7-b09d-0204f4d7fa3d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:47.544156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aefeff5-7ca6-4ca7-b09d-0204f4d7fa3d None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:47.545554 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 668/1347] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 15:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:47.550891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d524b-05ff-4c2d-97d8-62c3b38f5053 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:47.551345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d524b-05ff-4c2d-97d8-62c3b38f5053 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:47.551442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e3d524b-05ff-4c2d-97d8-62c3b38f5053 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:47.569906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9e3d524b-05ff-4c2d-97d8-62c3b38f5053 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:47.571096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9e3d524b-05ff-4c2d-97d8-62c3b38f5053 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:47.579386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 680/1348] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 15:59:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:59:47.615922 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefeff5-7ca6-4ca7-b09d-0204f4d7fa3d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 15:59:47.617978 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4aefeff5-7ca6-4ca7-b09d-0204f4d7fa3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:47.632559 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aefeff5-7ca6-4ca7-b09d-0204f4d7fa3d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 15:59:47.636559 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb64e2e-e75c-4c8e-ac76-ad346c116b5e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 15:59:47.637206 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb64e2e-e75c-4c8e-ac76-ad346c116b5e service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:47.637491 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb64e2e-e75c-4c8e-ac76-ad346c116b5e service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:47.637769 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb64e2e-e75c-4c8e-ac76-ad346c116b5e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:47.647087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb64e2e-e75c-4c8e-ac76-ad346c116b5e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 15:59:47.648585 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb64e2e-e75c-4c8e-ac76-ad346c116b5e service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 15:59:47.658089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 669/1349] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 15:59:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 15:59:52.917520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69825daa-a58e-4f63-9d88-094cbfd39c3a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:52.918377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69825daa-a58e-4f63-9d88-094cbfd39c3a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:52.918790 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69825daa-a58e-4f63-9d88-094cbfd39c3a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:52.921031 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 681/1350] 10.222.0.85 () {54 vars in 921 bytes} [Wed May 25 15:59:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:52.930999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9df5c-e1ee-4d2b-bebb-89d00a331cfc None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:52.931798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9df5c-e1ee-4d2b-bebb-89d00a331cfc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:52.932364 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa9df5c-e1ee-4d2b-bebb-89d00a331cfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:53.068651 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-eaa9df5c-e1ee-4d2b-bebb-89d00a331cfc None None] MFA Rules not processed for user `82b738f1b8c64590955a640c461b4ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:53.069841 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-eaa9df5c-e1ee-4d2b-bebb-89d00a331cfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:53.198966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 670/1351] 10.222.0.85 () {58 vars in 1018 bytes} [Wed May 25 15:59:52 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:59:57.139228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65090ea4-80db-4366-9062-b8c8ee7a98c8 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:57.140181 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65090ea4-80db-4366-9062-b8c8ee7a98c8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:57.140181 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65090ea4-80db-4366-9062-b8c8ee7a98c8 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:57.141378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 682/1352] 10.222.0.85 () {54 vars in 921 bytes} [Wed May 25 15:59:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:57.146232 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05382824-6e45-430a-8649-53b6a793d63b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:57.146695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05382824-6e45-430a-8649-53b6a793d63b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:57.147036 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05382824-6e45-430a-8649-53b6a793d63b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:57.166547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-05382824-6e45-430a-8649-53b6a793d63b None None] MFA Rules not processed for user `82b738f1b8c64590955a640c461b4ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:57.167738 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-05382824-6e45-430a-8649-53b6a793d63b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:57.177633 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 671/1353] 10.222.0.85 () {58 vars in 1018 bytes} [Wed May 25 15:59:57 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 15:59:57.564628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=13 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 15:59:58.741196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab666f2-8ece-42b4-8116-772e52e30e6a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:58.741835 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab666f2-8ece-42b4-8116-772e52e30e6a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:58.741835 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bab666f2-8ece-42b4-8116-772e52e30e6a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:58.743324 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 683/1354] 10.222.0.85 () {54 vars in 921 bytes} [Wed May 25 15:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 15:59:58.748681 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a44d62-aa1b-4963-ba41-b6a00b2e86e0 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 15:59:58.749110 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a44d62-aa1b-4963-ba41-b6a00b2e86e0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 15:59:58.749189 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a44d62-aa1b-4963-ba41-b6a00b2e86e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 15:59:58.770304 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-72a44d62-aa1b-4963-ba41-b6a00b2e86e0 None None] MFA Rules not processed for user `82b738f1b8c64590955a640c461b4ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 15:59:58.771630 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-72a44d62-aa1b-4963-ba41-b6a00b2e86e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 15:59:58.782532 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 672/1355] 10.222.0.85 () {58 vars in 1018 bytes} [Wed May 25 15:59:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:02.988823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62621284-0c40-459e-8a3e-e40ed2a36765 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:02.989341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62621284-0c40-459e-8a3e-e40ed2a36765 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:02.989441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62621284-0c40-459e-8a3e-e40ed2a36765 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:02.991057 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 684/1356] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:02.996289 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9352c6-1bc3-4a72-b348-51ea70e9e3f9 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:02.996752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9352c6-1bc3-4a72-b348-51ea70e9e3f9 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:02.996913 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9352c6-1bc3-4a72-b348-51ea70e9e3f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:03.014823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-cf9352c6-1bc3-4a72-b348-51ea70e9e3f9 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:03.016060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-cf9352c6-1bc3-4a72-b348-51ea70e9e3f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:03.024187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 673/1357] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:00:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:03.078253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c14844-96e4-48f2-9732-9cdb0d8d5255 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:03.078554 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c14844-96e4-48f2-9732-9cdb0d8d5255 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:03.078975 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c14844-96e4-48f2-9732-9cdb0d8d5255 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:03.080377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 685/1358] 10.222.0.85 () {54 vars in 961 bytes} [Wed May 25 16:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:03.088777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34daf82c-7b15-4127-b180-cc8d1acf09fc None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:03.089265 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34daf82c-7b15-4127-b180-cc8d1acf09fc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:03.089882 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34daf82c-7b15-4127-b180-cc8d1acf09fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:03.137208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:00:03.147418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-34daf82c-7b15-4127-b180-cc8d1acf09fc None None] MFA Rules not processed for user `4751abc9f67947b3ad17ba2b78ce2c12`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:03.148840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-34daf82c-7b15-4127-b180-cc8d1acf09fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:03.241066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 674/1359] 10.222.0.85 () {58 vars in 1058 bytes} [Wed May 25 16:00:03 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:03.249073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c14844-96e4-48f2-9732-9cdb0d8d5255 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:03.265032 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c14844-96e4-48f2-9732-9cdb0d8d5255 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:03.270968 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8500e83-7a7c-4391-8eae-5c7c8c3a0f50 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:03.271705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8500e83-7a7c-4391-8eae-5c7c8c3a0f50 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:03.272038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8500e83-7a7c-4391-8eae-5c7c8c3a0f50 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:03.272526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8500e83-7a7c-4391-8eae-5c7c8c3a0f50 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:03.282060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8500e83-7a7c-4391-8eae-5c7c8c3a0f50 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:03.283730 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8500e83-7a7c-4391-8eae-5c7c8c3a0f50 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:03.294421 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 686/1360] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:05.721679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e0908-62ff-40e3-86df-dfe986534f95 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:05.722273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e0908-62ff-40e3-86df-dfe986534f95 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:05.722273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122e0908-62ff-40e3-86df-dfe986534f95 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:05.723877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 675/1361] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:05.729150 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6e6ae-a6d0-4f41-ba9f-a16534fab6e6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:05.730513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6e6ae-a6d0-4f41-ba9f-a16534fab6e6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:05.731525 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b6e6ae-a6d0-4f41-ba9f-a16534fab6e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:05.750981 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-48b6e6ae-a6d0-4f41-ba9f-a16534fab6e6 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:05.752240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-48b6e6ae-a6d0-4f41-ba9f-a16534fab6e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:05.760522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 687/1362] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:00:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:05.805993 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e0908-62ff-40e3-86df-dfe986534f95 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:05.822422 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122e0908-62ff-40e3-86df-dfe986534f95 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:05.828253 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff00a2-a7b7-432f-b25e-3ac9186a99a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:05.828896 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff00a2-a7b7-432f-b25e-3ac9186a99a9 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:05.829186 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff00a2-a7b7-432f-b25e-3ac9186a99a9 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:05.829469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff00a2-a7b7-432f-b25e-3ac9186a99a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:05.839113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff00a2-a7b7-432f-b25e-3ac9186a99a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:05.840727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff00a2-a7b7-432f-b25e-3ac9186a99a9 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:05.851389 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 676/1363] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:08.397343 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190afd68-6ddb-4f5d-b378-9bc59ddbab1a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:08.398265 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190afd68-6ddb-4f5d-b378-9bc59ddbab1a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:08.398557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190afd68-6ddb-4f5d-b378-9bc59ddbab1a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:08.400188 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 688/1364] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:00:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:08.405380 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48615e03-0441-439c-b30e-f760fbf7a045 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:08.405816 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48615e03-0441-439c-b30e-f760fbf7a045 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:08.405946 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48615e03-0441-439c-b30e-f760fbf7a045 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:08.429995 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-48615e03-0441-439c-b30e-f760fbf7a045 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:08.431216 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-48615e03-0441-439c-b30e-f760fbf7a045 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:08.442429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 677/1365] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:00:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:08.487452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd48f90-171b-48ed-b499-1f5cc10ed198 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:08.487745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd48f90-171b-48ed-b499-1f5cc10ed198 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:08.488058 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dd48f90-171b-48ed-b499-1f5cc10ed198 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:08.489420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 689/1366] 10.222.0.85 () {54 vars in 961 bytes} [Wed May 25 16:00:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:08.494727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ac8af-7ab7-4d81-b99b-c9a91c6a395c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:08.495028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ac8af-7ab7-4d81-b99b-c9a91c6a395c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:08.495315 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3ac8af-7ab7-4d81-b99b-c9a91c6a395c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:08.517549 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-eb3ac8af-7ab7-4d81-b99b-c9a91c6a395c None None] MFA Rules not processed for user `4751abc9f67947b3ad17ba2b78ce2c12`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:08.518730 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-eb3ac8af-7ab7-4d81-b99b-c9a91c6a395c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:08.529342 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 678/1367] 10.222.0.85 () {58 vars in 1058 bytes} [Wed May 25 16:00:08 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:08.537987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd48f90-171b-48ed-b499-1f5cc10ed198 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:08.552426 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dd48f90-171b-48ed-b499-1f5cc10ed198 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:08.557788 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96353daa-d82a-411a-81ec-29718ab67e6f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:08.558371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96353daa-d82a-411a-81ec-29718ab67e6f service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:08.558826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96353daa-d82a-411a-81ec-29718ab67e6f service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:08.559271 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96353daa-d82a-411a-81ec-29718ab67e6f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:08.573356 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96353daa-d82a-411a-81ec-29718ab67e6f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:08.576512 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96353daa-d82a-411a-81ec-29718ab67e6f service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:08.589909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 690/1368] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:00:10.858423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f98898-fcdd-4670-87fd-5757f441b3ed None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:10.858997 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f98898-fcdd-4670-87fd-5757f441b3ed None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:10.858997 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f98898-fcdd-4670-87fd-5757f441b3ed None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:10.860395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 679/1369] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:00:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:10.865933 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bb236-d62b-4a8e-ab10-7c17b694a3c9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:10.866372 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bb236-d62b-4a8e-ab10-7c17b694a3c9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:10.866478 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171bb236-d62b-4a8e-ab10-7c17b694a3c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:10.884810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-171bb236-d62b-4a8e-ab10-7c17b694a3c9 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:10.886055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-171bb236-d62b-4a8e-ab10-7c17b694a3c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:10.895628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 691/1370] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:00:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:10.941831 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f98898-fcdd-4670-87fd-5757f441b3ed None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:10.957994 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f98898-fcdd-4670-87fd-5757f441b3ed None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:10.964448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e10769f-13ca-4138-a984-19a4e0b0be0c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:10.965210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10769f-13ca-4138-a984-19a4e0b0be0c service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:10.965738 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10769f-13ca-4138-a984-19a4e0b0be0c service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:10.966087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e10769f-13ca-4138-a984-19a4e0b0be0c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:10.975923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10769f-13ca-4138-a984-19a4e0b0be0c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:10.977624 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e10769f-13ca-4138-a984-19a4e0b0be0c service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:10.988240 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 680/1371] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:11.124322 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1975a-ee90-469a-ba4d-c7432b7518d5 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:11.124610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1975a-ee90-469a-ba4d-c7432b7518d5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:11.124865 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef1975a-ee90-469a-ba4d-c7432b7518d5 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:11.126587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 692/1372] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:00:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:11.133005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66377dc-0a6a-4cf2-b7e8-67779cab1c77 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:11.133359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66377dc-0a6a-4cf2-b7e8-67779cab1c77 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:11.133638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66377dc-0a6a-4cf2-b7e8-67779cab1c77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:11.153158 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b66377dc-0a6a-4cf2-b7e8-67779cab1c77 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:11.154399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b66377dc-0a6a-4cf2-b7e8-67779cab1c77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:11.163004 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 681/1373] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:00:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:11.474755 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52363456-3765-41e5-9b86-61b5f3b84952 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:11.475054 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52363456-3765-41e5-9b86-61b5f3b84952 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:11.475343 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52363456-3765-41e5-9b86-61b5f3b84952 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:11.476855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 693/1374] 10.222.0.85 () {54 vars in 963 bytes} [Wed May 25 16:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:11.484056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0de2ed-6135-4d5c-a811-1ed168d0c91d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:11.484360 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0de2ed-6135-4d5c-a811-1ed168d0c91d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:11.484647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0de2ed-6135-4d5c-a811-1ed168d0c91d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:11.538275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8f0de2ed-6135-4d5c-a811-1ed168d0c91d None None] MFA Rules not processed for user `f81aecdc87d84501ba94b208405b51dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:11.539579 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8f0de2ed-6135-4d5c-a811-1ed168d0c91d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:11.604440 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 682/1375] 10.222.0.85 () {58 vars in 1060 bytes} [Wed May 25 16:00:11 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:11.613229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52363456-3765-41e5-9b86-61b5f3b84952 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:11.624987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52363456-3765-41e5-9b86-61b5f3b84952 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:11.628673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd67fc5-d042-422a-9ef2-fd44f843ede6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:11.629447 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd67fc5-d042-422a-9ef2-fd44f843ede6 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:11.629811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd67fc5-d042-422a-9ef2-fd44f843ede6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:11.630192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd67fc5-d042-422a-9ef2-fd44f843ede6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:11.640265 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd67fc5-d042-422a-9ef2-fd44f843ede6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:11.641823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd67fc5-d042-422a-9ef2-fd44f843ede6 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:11.652014 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 694/1376] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:13.264822 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb690039-8b5e-4889-be4c-73c9d9954867 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:13.265374 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb690039-8b5e-4889-be4c-73c9d9954867 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:13.265374 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb690039-8b5e-4889-be4c-73c9d9954867 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:13.266942 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 683/1377] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:00:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:13.273197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41b034-b878-459f-93ba-b85b37ccb377 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:13.273788 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41b034-b878-459f-93ba-b85b37ccb377 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:13.273788 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb41b034-b878-459f-93ba-b85b37ccb377 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:13.293370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-fb41b034-b878-459f-93ba-b85b37ccb377 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:13.294592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fb41b034-b878-459f-93ba-b85b37ccb377 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:13.303203 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 695/1378] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:00:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:13.348443 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb690039-8b5e-4889-be4c-73c9d9954867 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:13.364208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb690039-8b5e-4889-be4c-73c9d9954867 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:13.370048 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c2dd14-06bb-4d51-8215-fce190153e71 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:13.370780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2dd14-06bb-4d51-8215-fce190153e71 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:13.371073 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2dd14-06bb-4d51-8215-fce190153e71 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:13.371362 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c2dd14-06bb-4d51-8215-fce190153e71 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:13.380903 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2dd14-06bb-4d51-8215-fce190153e71 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:13.382535 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c2dd14-06bb-4d51-8215-fce190153e71 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:13.387136 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:00:13.394758 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 684/1379] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:15.641301 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7879bb6-06d5-429d-a949-365659b4c38e None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:15.641872 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7879bb6-06d5-429d-a949-365659b4c38e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:15.641872 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7879bb6-06d5-429d-a949-365659b4c38e None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:15.643233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 696/1380] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:00:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:15.648558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553bd91-9524-4519-8420-2986addb8550 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:15.648977 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553bd91-9524-4519-8420-2986addb8550 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:15.649104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6553bd91-9524-4519-8420-2986addb8550 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:15.671399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6553bd91-9524-4519-8420-2986addb8550 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:15.671399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6553bd91-9524-4519-8420-2986addb8550 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:15.678406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 685/1381] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:00:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:15.721835 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7879bb6-06d5-429d-a949-365659b4c38e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:15.737738 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7879bb6-06d5-429d-a949-365659b4c38e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:15.744021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce78628-d2f9-46ff-b998-4f9c6e5830f5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:15.744679 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce78628-d2f9-46ff-b998-4f9c6e5830f5 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:15.744965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce78628-d2f9-46ff-b998-4f9c6e5830f5 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:15.745247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce78628-d2f9-46ff-b998-4f9c6e5830f5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:15.754446 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce78628-d2f9-46ff-b998-4f9c6e5830f5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:15.756077 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce78628-d2f9-46ff-b998-4f9c6e5830f5 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:15.766144 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 697/1382] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:18.009538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaa451-7224-4f13-8e84-5fde7be560f7 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:18.010153 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaa451-7224-4f13-8e84-5fde7be560f7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:18.010153 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66aaa451-7224-4f13-8e84-5fde7be560f7 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:18.011625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 686/1383] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:00:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:18.018499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631c42d-338d-465b-b5de-97cf3de239f6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:18.019077 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631c42d-338d-465b-b5de-97cf3de239f6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:18.019077 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b631c42d-338d-465b-b5de-97cf3de239f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:18.040318 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b631c42d-338d-465b-b5de-97cf3de239f6 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:18.041624 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b631c42d-338d-465b-b5de-97cf3de239f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:18.051129 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 698/1384] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:00:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:18.097485 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaa451-7224-4f13-8e84-5fde7be560f7 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:18.113690 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66aaa451-7224-4f13-8e84-5fde7be560f7 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:18.119993 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f483b6c0-604b-4065-824e-dd367f76918c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:18.120727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483b6c0-604b-4065-824e-dd367f76918c service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:18.121083 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483b6c0-604b-4065-824e-dd367f76918c service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:18.121428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f483b6c0-604b-4065-824e-dd367f76918c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:18.130951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483b6c0-604b-4065-824e-dd367f76918c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:18.132975 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f483b6c0-604b-4065-824e-dd367f76918c service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:18.143508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 687/1385] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:20.331699 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a2d08-2c76-4cf2-9e8c-cad302f593af None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:20.332370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a2d08-2c76-4cf2-9e8c-cad302f593af None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:20.332370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f66a2d08-2c76-4cf2-9e8c-cad302f593af None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:20.333654 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 699/1386] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:20.339254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7062df-3b47-427a-acca-341ae8e341d2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:20.339725 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7062df-3b47-427a-acca-341ae8e341d2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:20.339855 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7062df-3b47-427a-acca-341ae8e341d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:20.358662 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-9e7062df-3b47-427a-acca-341ae8e341d2 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:20.359963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9e7062df-3b47-427a-acca-341ae8e341d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:20.368624 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 688/1387] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:00:20.412263 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a2d08-2c76-4cf2-9e8c-cad302f593af None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:20.428805 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f66a2d08-2c76-4cf2-9e8c-cad302f593af None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:20.434768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9e24fc-5b4a-4463-be95-efd20b83c706 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:20.435766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e24fc-5b4a-4463-be95-efd20b83c706 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:20.436050 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e24fc-5b4a-4463-be95-efd20b83c706 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:20.436340 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9e24fc-5b4a-4463-be95-efd20b83c706 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:20.445081 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e24fc-5b4a-4463-be95-efd20b83c706 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:20.446702 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9e24fc-5b4a-4463-be95-efd20b83c706 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:20.456361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 700/1388] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:22.685946 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20c9a8-eb63-4fe9-a74a-cec542b3eafd None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:22.686660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20c9a8-eb63-4fe9-a74a-cec542b3eafd None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:22.686660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d20c9a8-eb63-4fe9-a74a-cec542b3eafd None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:22.687968 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 689/1389] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:00:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:22.693611 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76e7ee-3881-49d7-94f9-9af9c52c0d70 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:22.694061 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76e7ee-3881-49d7-94f9-9af9c52c0d70 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:22.694177 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac76e7ee-3881-49d7-94f9-9af9c52c0d70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:22.713145 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ac76e7ee-3881-49d7-94f9-9af9c52c0d70 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:22.714407 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ac76e7ee-3881-49d7-94f9-9af9c52c0d70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:22.722906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 701/1390] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:00:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:22.767099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20c9a8-eb63-4fe9-a74a-cec542b3eafd None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:22.783999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d20c9a8-eb63-4fe9-a74a-cec542b3eafd None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:22.789732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b67209-58e3-48b4-a161-45751739b100 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:22.790393 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b67209-58e3-48b4-a161-45751739b100 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:22.790679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b67209-58e3-48b4-a161-45751739b100 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:22.790954 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b67209-58e3-48b4-a161-45751739b100 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:22.799479 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b67209-58e3-48b4-a161-45751739b100 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:22.801131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b67209-58e3-48b4-a161-45751739b100 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:22.810912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 690/1391] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:25.044060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669aa54-5dc4-4b3f-80a6-ae290b6cba68 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:25.044676 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669aa54-5dc4-4b3f-80a6-ae290b6cba68 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:25.044797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669aa54-5dc4-4b3f-80a6-ae290b6cba68 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:25.046571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 702/1392] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:00:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:25.053292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5ffe2-5756-43d3-9934-0aed7e7c7ea7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:25.054000 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5ffe2-5756-43d3-9934-0aed7e7c7ea7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:25.054000 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5ffe2-5756-43d3-9934-0aed7e7c7ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:25.076378 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c7c5ffe2-5756-43d3-9934-0aed7e7c7ea7 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:25.077664 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c7c5ffe2-5756-43d3-9934-0aed7e7c7ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:25.086398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 691/1393] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:25.130767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669aa54-5dc4-4b3f-80a6-ae290b6cba68 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:25.146197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669aa54-5dc4-4b3f-80a6-ae290b6cba68 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:25.152294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ccb6889-8a4f-49ab-8864-67129f86bb0e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:25.152948 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb6889-8a4f-49ab-8864-67129f86bb0e service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:25.153223 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb6889-8a4f-49ab-8864-67129f86bb0e service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:25.153504 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ccb6889-8a4f-49ab-8864-67129f86bb0e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:25.162397 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb6889-8a4f-49ab-8864-67129f86bb0e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:25.164030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ccb6889-8a4f-49ab-8864-67129f86bb0e service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:25.174253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 703/1394] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:27.217555 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0627e7-1a20-40de-a27a-7ab750a246ab None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:27.218221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0627e7-1a20-40de-a27a-7ab750a246ab None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:27.218325 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0627e7-1a20-40de-a27a-7ab750a246ab None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:27.219979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 692/1395] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:27.225757 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6d106-6d6b-4033-9108-1a98d5b259a8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:27.226367 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6d106-6d6b-4033-9108-1a98d5b259a8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:27.226367 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc6d106-6d6b-4033-9108-1a98d5b259a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:27.245414 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dcc6d106-6d6b-4033-9108-1a98d5b259a8 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:27.246631 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dcc6d106-6d6b-4033-9108-1a98d5b259a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:27.255074 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 704/1396] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:27.310403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce8810-c606-4688-945e-2fa06d28d2ed None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:27.310693 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce8810-c606-4688-945e-2fa06d28d2ed None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:27.310973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ce8810-c606-4688-945e-2fa06d28d2ed None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:27.312402 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 693/1397] 10.222.0.85 () {54 vars in 962 bytes} [Wed May 25 16:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:27.317390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3e881-24f0-4ddb-a9f3-d5f5fd461d43 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:27.317685 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3e881-24f0-4ddb-a9f3-d5f5fd461d43 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:27.317967 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ce3e881-24f0-4ddb-a9f3-d5f5fd461d43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:27.370230 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7ce3e881-24f0-4ddb-a9f3-d5f5fd461d43 None None] MFA Rules not processed for user `196dc406a95444c39babc50d59f9eb6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:27.371478 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7ce3e881-24f0-4ddb-a9f3-d5f5fd461d43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:27.395274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52171eb-a43f-4abd-9f3d-8627b2164cdc None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:27.395572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52171eb-a43f-4abd-9f3d-8627b2164cdc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:27.395856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52171eb-a43f-4abd-9f3d-8627b2164cdc None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:27.397275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 694/1398] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:00:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:27.402409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68639da3-d1ad-4b25-a25d-b34df3a3374d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:27.402701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68639da3-d1ad-4b25-a25d-b34df3a3374d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:27.402982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68639da3-d1ad-4b25-a25d-b34df3a3374d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:27.420448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-68639da3-d1ad-4b25-a25d-b34df3a3374d None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:27.421654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-68639da3-d1ad-4b25-a25d-b34df3a3374d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:27.429816 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 695/1399] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:00:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:00:27.435592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 705/1400] 10.222.0.85 () {58 vars in 1059 bytes} [Wed May 25 16:00:27 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:27.443300 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68639da3-d1ad-4b25-a25d-b34df3a3374d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:27.452620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68639da3-d1ad-4b25-a25d-b34df3a3374d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:27.456189 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91c981be-bd96-4bd9-b5e1-6d8688227f5e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '196dc406a95444c39babc50d59f9eb6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:27.456826 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c981be-bd96-4bd9-b5e1-6d8688227f5e service glance] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:27.457109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c981be-bd96-4bd9-b5e1-6d8688227f5e service glance] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:27.457387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91c981be-bd96-4bd9-b5e1-6d8688227f5e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:27.466566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c981be-bd96-4bd9-b5e1-6d8688227f5e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:27.468077 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91c981be-bd96-4bd9-b5e1-6d8688227f5e service glance] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:27.473932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3e881-24f0-4ddb-a9f3-d5f5fd461d43 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:27.477538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 696/1401] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:27.487039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ce3e881-24f0-4ddb-a9f3-d5f5fd461d43 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:27.492585 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cef6a16-b330-42bf-9215-fdcfbf81736f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:27.493234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef6a16-b330-42bf-9215-fdcfbf81736f service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:27.493519 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef6a16-b330-42bf-9215-fdcfbf81736f service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:27.493799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cef6a16-b330-42bf-9215-fdcfbf81736f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:27.503528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef6a16-b330-42bf-9215-fdcfbf81736f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:27.504871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cef6a16-b330-42bf-9215-fdcfbf81736f service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:27.513944 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 706/1402] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:28.017738 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b7680-8b98-4b1b-b83b-d8c6c70226b9 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:28.018031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b7680-8b98-4b1b-b83b-d8c6c70226b9 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:28.018320 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb3b7680-8b98-4b1b-b83b-d8c6c70226b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:28.068399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-cb3b7680-8b98-4b1b-b83b-d8c6c70226b9 None None] MFA Rules not processed for user `066de1f11f724eee9df0377beb58d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:28.069638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-cb3b7680-8b98-4b1b-b83b-d8c6c70226b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:28.134259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 697/1403] 10.222.0.85 () {58 vars in 1009 bytes} [Wed May 25 16:00:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:28.177887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9aa62-4420-44d8-b4f0-695f6ea108c1 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:28.178187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9aa62-4420-44d8-b4f0-695f6ea108c1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:28.178491 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c9aa62-4420-44d8-b4f0-695f6ea108c1 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:28.179968 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 707/1404] 10.222.0.85 () {54 vars in 942 bytes} [Wed May 25 16:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:28.186598 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d0f56-752a-4269-b45f-5cec6eaaa1d0 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:28.186897 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d0f56-752a-4269-b45f-5cec6eaaa1d0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:28.187182 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654d0f56-752a-4269-b45f-5cec6eaaa1d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:28.236241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-654d0f56-752a-4269-b45f-5cec6eaaa1d0 None None] MFA Rules not processed for user `77fda27641c84fccb68a5cdaf4e8dc4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:28.237448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-654d0f56-752a-4269-b45f-5cec6eaaa1d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:28.270727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:00:28.312077 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 698/1405] 10.222.0.85 () {58 vars in 1039 bytes} [Wed May 25 16:00:28 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:28.322014 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9aa62-4420-44d8-b4f0-695f6ea108c1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:28.329543 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:00:28.336449 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c9aa62-4420-44d8-b4f0-695f6ea108c1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:28.341578 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77eb85c7-1167-4c9d-b2b7-9b5f5e39ce91 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77fda27641c84fccb68a5cdaf4e8dc4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:28.342232 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb85c7-1167-4c9d-b2b7-9b5f5e39ce91 service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:28.342519 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb85c7-1167-4c9d-b2b7-9b5f5e39ce91 service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:28.342802 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77eb85c7-1167-4c9d-b2b7-9b5f5e39ce91 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:28.351228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb85c7-1167-4c9d-b2b7-9b5f5e39ce91 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:28.352856 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77eb85c7-1167-4c9d-b2b7-9b5f5e39ce91 service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:28.361055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 708/1406] 10.222.0.85 () {60 vars in 1384 bytes} [Wed May 25 16:00:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:00:29.755081 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b89aff-95d3-4f07-a469-7f19bea0c699 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:29.755644 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b89aff-95d3-4f07-a469-7f19bea0c699 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:29.755644 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b89aff-95d3-4f07-a469-7f19bea0c699 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:29.757073 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 699/1407] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:29.762358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f139d-0ac2-410f-b312-2b6e47a6eccb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:29.762822 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f139d-0ac2-410f-b312-2b6e47a6eccb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:29.762919 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91f139d-0ac2-410f-b312-2b6e47a6eccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:29.781910 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b91f139d-0ac2-410f-b312-2b6e47a6eccb None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:29.783119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b91f139d-0ac2-410f-b312-2b6e47a6eccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:29.793964 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 709/1408] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:29.837325 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b89aff-95d3-4f07-a469-7f19bea0c699 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:29.853696 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b89aff-95d3-4f07-a469-7f19bea0c699 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:29.861717 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-596a8896-32b6-425a-8542-e533742b3a73 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:29.862499 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a8896-32b6-425a-8542-e533742b3a73 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:29.862853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a8896-32b6-425a-8542-e533742b3a73 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:29.863199 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-596a8896-32b6-425a-8542-e533742b3a73 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:29.873626 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a8896-32b6-425a-8542-e533742b3a73 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:29.875477 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-596a8896-32b6-425a-8542-e533742b3a73 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:29.886393 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 700/1409] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:39.775775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=8 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:00:39.881977 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=21 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:00:51.326302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d84c19-f937-4120-abeb-dc442358dfa3 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:51.326830 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d84c19-f937-4120-abeb-dc442358dfa3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:51.326938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d84c19-f937-4120-abeb-dc442358dfa3 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:51.328344 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 710/1410] 10.222.0.85 () {54 vars in 918 bytes} [Wed May 25 16:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:51.333874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2df4e8-95be-48f8-a26a-b26cd61fef21 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:51.334385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2df4e8-95be-48f8-a26a-b26cd61fef21 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:51.334385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2df4e8-95be-48f8-a26a-b26cd61fef21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:51.353445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-af2df4e8-95be-48f8-a26a-b26cd61fef21 None None] MFA Rules not processed for user `82b738f1b8c64590955a640c461b4ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:51.354672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-af2df4e8-95be-48f8-a26a-b26cd61fef21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:51.365341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 701/1411] 10.222.0.85 () {58 vars in 1015 bytes} [Wed May 25 16:00:51 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:00:51.380090 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21ad64-8278-4c98-bdfc-8c37c4c22ed9 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:51.380384 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21ad64-8278-4c98-bdfc-8c37c4c22ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:51.380659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e21ad64-8278-4c98-bdfc-8c37c4c22ed9 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:51.382002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 711/1412] 10.222.0.85 () {54 vars in 918 bytes} [Wed May 25 16:00:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:51.386638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c884f99-9f34-4763-b947-794a1de1becd None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:51.386932 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c884f99-9f34-4763-b947-794a1de1becd None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:51.387205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c884f99-9f34-4763-b947-794a1de1becd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:51.405839 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-2c884f99-9f34-4763-b947-794a1de1becd None None] MFA Rules not processed for user `4751abc9f67947b3ad17ba2b78ce2c12`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:51.407010 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2c884f99-9f34-4763-b947-794a1de1becd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:51.417040 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 702/1413] 10.222.0.85 () {58 vars in 1015 bytes} [Wed May 25 16:00:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:54.080300 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c92cd-8447-4e53-831f-aca1c2b2622a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:54.081271 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c92cd-8447-4e53-831f-aca1c2b2622a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:54.081271 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f9c92cd-8447-4e53-831f-aca1c2b2622a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:54.083338 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 712/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:00:54 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:54.092857 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d75a2-86b3-4b5f-a130-33cd79e79fcf None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:54.093296 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d75a2-86b3-4b5f-a130-33cd79e79fcf None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:54.093431 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89d75a2-86b3-4b5f-a130-33cd79e79fcf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:54.115722 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-a89d75a2-86b3-4b5f-a130-33cd79e79fcf None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:54.116944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a89d75a2-86b3-4b5f-a130-33cd79e79fcf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:54.125322 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 703/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:00:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:54.168776 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c92cd-8447-4e53-831f-aca1c2b2622a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:54.186528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f9c92cd-8447-4e53-831f-aca1c2b2622a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:54.192415 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1807e31e-cca2-48dd-9d3f-b34584c87b77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:54.193060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807e31e-cca2-48dd-9d3f-b34584c87b77 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:54.193342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807e31e-cca2-48dd-9d3f-b34584c87b77 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:54.193620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1807e31e-cca2-48dd-9d3f-b34584c87b77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:54.201748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807e31e-cca2-48dd-9d3f-b34584c87b77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:54.203354 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1807e31e-cca2-48dd-9d3f-b34584c87b77 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:54.212618 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 713/1416] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:55.186174 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619c154-f9dd-4e8b-a248-3315f1433927 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:55.186755 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619c154-f9dd-4e8b-a248-3315f1433927 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:55.186755 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b619c154-f9dd-4e8b-a248-3315f1433927 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:55.188202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 704/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:55.194517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2b703-9d0a-4ce8-8b4a-5f6ded7f544b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:55.195251 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2b703-9d0a-4ce8-8b4a-5f6ded7f544b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:55.195408 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e2b703-9d0a-4ce8-8b4a-5f6ded7f544b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:55.218341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-36e2b703-9d0a-4ce8-8b4a-5f6ded7f544b None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:55.219545 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-36e2b703-9d0a-4ce8-8b4a-5f6ded7f544b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:55.228392 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 714/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:55.386562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619c154-f9dd-4e8b-a248-3315f1433927 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:55.409836 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b619c154-f9dd-4e8b-a248-3315f1433927 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:55.418021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ddc092-1444-45f9-a2ba-549b4d2d5a65 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:55.418982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddc092-1444-45f9-a2ba-549b4d2d5a65 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:55.419392 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddc092-1444-45f9-a2ba-549b4d2d5a65 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:55.419759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ddc092-1444-45f9-a2ba-549b4d2d5a65 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:55.433505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ddc092-1444-45f9-a2ba-549b4d2d5a65 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:55.436026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24ddc092-1444-45f9-a2ba-549b4d2d5a65 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:55.450252 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 705/1419] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:56.375727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab70337-277e-4e47-93b9-e265e303f29c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:56.376327 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab70337-277e-4e47-93b9-e265e303f29c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:56.376327 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab70337-277e-4e47-93b9-e265e303f29c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:56.377697 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 715/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:00:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:56.383980 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fb564-c625-4c9c-a3f5-d93de1086eb5 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:56.384484 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fb564-c625-4c9c-a3f5-d93de1086eb5 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:56.384654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786fb564-c625-4c9c-a3f5-d93de1086eb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:56.405702 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-786fb564-c625-4c9c-a3f5-d93de1086eb5 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:56.406899 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-786fb564-c625-4c9c-a3f5-d93de1086eb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:56.416571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 706/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:56.573573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab70337-277e-4e47-93b9-e265e303f29c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:56.590141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab70337-277e-4e47-93b9-e265e303f29c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:56.596282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2252f544-7cef-4841-b99d-d7d80318f327 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:56.596927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252f544-7cef-4841-b99d-d7d80318f327 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:56.597220 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252f544-7cef-4841-b99d-d7d80318f327 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:56.597523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2252f544-7cef-4841-b99d-d7d80318f327 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:56.606085 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252f544-7cef-4841-b99d-d7d80318f327 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:56.607702 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2252f544-7cef-4841-b99d-d7d80318f327 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:56.617146 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 716/1422] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:57.534070 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551a6dc-cce6-4d8d-9ccd-3a3675303fe7 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:57.534667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551a6dc-cce6-4d8d-9ccd-3a3675303fe7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:57.534667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9551a6dc-cce6-4d8d-9ccd-3a3675303fe7 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:57.536608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 707/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:00:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:57.543686 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897aca0d-d60e-43dc-bf6c-0bc485674fe6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:57.544192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897aca0d-d60e-43dc-bf6c-0bc485674fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:57.544305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897aca0d-d60e-43dc-bf6c-0bc485674fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:57.565936 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-897aca0d-d60e-43dc-bf6c-0bc485674fe6 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:57.567085 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-897aca0d-d60e-43dc-bf6c-0bc485674fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:57.575599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 717/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:00:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:57.729324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551a6dc-cce6-4d8d-9ccd-3a3675303fe7 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:57.745665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9551a6dc-cce6-4d8d-9ccd-3a3675303fe7 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:57.755052 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca338a9b-3b83-4458-80f0-3bbc28d96725 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:57.756036 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca338a9b-3b83-4458-80f0-3bbc28d96725 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:57.756423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca338a9b-3b83-4458-80f0-3bbc28d96725 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:57.756803 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca338a9b-3b83-4458-80f0-3bbc28d96725 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:57.766558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca338a9b-3b83-4458-80f0-3bbc28d96725 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:57.768211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca338a9b-3b83-4458-80f0-3bbc28d96725 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:57.778213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 708/1425] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:58.672555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe9f37-ba06-4708-95e8-6eb2fde3b60b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:58.673539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe9f37-ba06-4708-95e8-6eb2fde3b60b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:58.673903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16fe9f37-ba06-4708-95e8-6eb2fde3b60b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:58.675288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 718/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:00:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:58.681657 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db00608-7cc1-4e74-a9aa-5fb3bef38144 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:58.682097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db00608-7cc1-4e74-a9aa-5fb3bef38144 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:58.682213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db00608-7cc1-4e74-a9aa-5fb3bef38144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:58.707080 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-9db00608-7cc1-4e74-a9aa-5fb3bef38144 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:58.709002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9db00608-7cc1-4e74-a9aa-5fb3bef38144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:58.719947 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 709/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:00:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:58.871661 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe9f37-ba06-4708-95e8-6eb2fde3b60b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:58.896488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16fe9f37-ba06-4708-95e8-6eb2fde3b60b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:00:58.903861 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e1e45d-c920-43f7-a63a-28b83ef9f443 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:00:58.904501 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1e45d-c920-43f7-a63a-28b83ef9f443 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:58.904790 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1e45d-c920-43f7-a63a-28b83ef9f443 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:58.905077 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e1e45d-c920-43f7-a63a-28b83ef9f443 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:58.913866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1e45d-c920-43f7-a63a-28b83ef9f443 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:00:58.915522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46e1e45d-c920-43f7-a63a-28b83ef9f443 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:00:58.925651 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 719/1428] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:00:59.777692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621689c2-8cfb-49ab-83d6-98103d6677b8 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:59.778281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621689c2-8cfb-49ab-83d6-98103d6677b8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:59.778281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621689c2-8cfb-49ab-83d6-98103d6677b8 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:59.779669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 710/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:00:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:00:59.784843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d09e7e-99e5-473f-9246-ed410c61d581 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:00:59.785290 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d09e7e-99e5-473f-9246-ed410c61d581 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:00:59.785606 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d09e7e-99e5-473f-9246-ed410c61d581 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:00:59.813060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-33d09e7e-99e5-473f-9246-ed410c61d581 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:00:59.814281 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-33d09e7e-99e5-473f-9246-ed410c61d581 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:00:59.823953 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 720/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:00:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:00:59.975957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621689c2-8cfb-49ab-83d6-98103d6677b8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:00:59.998120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621689c2-8cfb-49ab-83d6-98103d6677b8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:01:00.007609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0daa9ef-f164-4131-8c0d-81ae946e969a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:01:00.008450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daa9ef-f164-4131-8c0d-81ae946e969a service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:00.008808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daa9ef-f164-4131-8c0d-81ae946e969a service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:00.009224 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0daa9ef-f164-4131-8c0d-81ae946e969a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:00.018157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0daa9ef-f164-4131-8c0d-81ae946e969a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:01:00.019757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0daa9ef-f164-4131-8c0d-81ae946e969a service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:01:00.029604 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 711/1431] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:00:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:01:00.869311 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a090d-5228-4fd6-9e98-775586cd046a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:00.870221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a090d-5228-4fd6-9e98-775586cd046a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:00.870221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4a090d-5228-4fd6-9e98-775586cd046a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:00.872578 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 721/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:01:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:01:00.878861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06063787-e77a-4985-981d-375a51f70805 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:00.879303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06063787-e77a-4985-981d-375a51f70805 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:00.879481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06063787-e77a-4985-981d-375a51f70805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:00.900094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-06063787-e77a-4985-981d-375a51f70805 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:01:00.901319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-06063787-e77a-4985-981d-375a51f70805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:01:00.910124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 712/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:01:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:01:01.058571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a090d-5228-4fd6-9e98-775586cd046a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:01:01.075710 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4a090d-5228-4fd6-9e98-775586cd046a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:01:01.081829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e85089f-f63f-42e6-8222-5e2d2291ebad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:01:01.082467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85089f-f63f-42e6-8222-5e2d2291ebad service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:01.082752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85089f-f63f-42e6-8222-5e2d2291ebad service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:01.083208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e85089f-f63f-42e6-8222-5e2d2291ebad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:01.092025 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85089f-f63f-42e6-8222-5e2d2291ebad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:01:01.093638 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e85089f-f63f-42e6-8222-5e2d2291ebad service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:01:01.103344 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 722/1434] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:01:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:01:01.965975 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c60c2c-8876-4cf7-b037-f7af095ab5bb None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:01.966585 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c60c2c-8876-4cf7-b037-f7af095ab5bb None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:01.966585 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c60c2c-8876-4cf7-b037-f7af095ab5bb None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:01.967965 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 713/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:01:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:01:01.974181 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697c144-e844-4847-9f18-09cf252c4e72 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:01.974912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697c144-e844-4847-9f18-09cf252c4e72 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:01.975111 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2697c144-e844-4847-9f18-09cf252c4e72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:02.001097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2697c144-e844-4847-9f18-09cf252c4e72 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:01:02.002342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2697c144-e844-4847-9f18-09cf252c4e72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:01:02.011442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 723/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:01:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:01:02.163035 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c60c2c-8876-4cf7-b037-f7af095ab5bb None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:01:02.180211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c60c2c-8876-4cf7-b037-f7af095ab5bb None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:01:02.187367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c43c79-bdc1-464b-8e0b-5be056ef4467 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:01:02.188083 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c43c79-bdc1-464b-8e0b-5be056ef4467 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:02.188345 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c43c79-bdc1-464b-8e0b-5be056ef4467 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:02.188625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c43c79-bdc1-464b-8e0b-5be056ef4467 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:02.197049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c43c79-bdc1-464b-8e0b-5be056ef4467 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:01:02.198648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c43c79-bdc1-464b-8e0b-5be056ef4467 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:01:02.208225 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 714/1437] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:01:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:01:03.045735 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0109958-6859-48dd-8a0e-188af33f31b3 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:03.046605 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0109958-6859-48dd-8a0e-188af33f31b3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:03.046605 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0109958-6859-48dd-8a0e-188af33f31b3 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:03.048834 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 724/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:01:03 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:01:03.054010 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee1cb5-0a60-4b1f-ad9b-9c0ee7b568c0 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:03.054453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee1cb5-0a60-4b1f-ad9b-9c0ee7b568c0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:03.054549 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebee1cb5-0a60-4b1f-ad9b-9c0ee7b568c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:03.074981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ebee1cb5-0a60-4b1f-ad9b-9c0ee7b568c0 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:01:03.076212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ebee1cb5-0a60-4b1f-ad9b-9c0ee7b568c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:01:03.085027 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 715/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:01:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:01:03.243282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0109958-6859-48dd-8a0e-188af33f31b3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:01:03.260191 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0109958-6859-48dd-8a0e-188af33f31b3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:01:03.266687 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1f27861-33b2-4940-a4aa-1ee354a7f901 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:01:03.267328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f27861-33b2-4940-a4aa-1ee354a7f901 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:03.267615 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f27861-33b2-4940-a4aa-1ee354a7f901 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:03.267925 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1f27861-33b2-4940-a4aa-1ee354a7f901 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:03.276753 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f27861-33b2-4940-a4aa-1ee354a7f901 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:01:03.278365 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1f27861-33b2-4940-a4aa-1ee354a7f901 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:01:03.287847 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 725/1440] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:01:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:01:05.352277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0a1e8-00a0-4f0e-8a1c-2c763125e394 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:05.353170 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0a1e8-00a0-4f0e-8a1c-2c763125e394 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:05.353170 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71d0a1e8-00a0-4f0e-8a1c-2c763125e394 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:05.355439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 716/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:01:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:01:05.361119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebe6ff-7de0-496a-bf5d-cc9fd160d504 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:05.361593 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebe6ff-7de0-496a-bf5d-cc9fd160d504 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:05.361700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cebe6ff-7de0-496a-bf5d-cc9fd160d504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:05.381356 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6cebe6ff-7de0-496a-bf5d-cc9fd160d504 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:01:05.382541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6cebe6ff-7de0-496a-bf5d-cc9fd160d504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:01:05.390899 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 726/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:01:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:01:05.433132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0a1e8-00a0-4f0e-8a1c-2c763125e394 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:01:05.449907 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71d0a1e8-00a0-4f0e-8a1c-2c763125e394 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:01:05.454005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abce611e-5607-4aba-9378-14a702f5c8ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:01:05.454660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce611e-5607-4aba-9378-14a702f5c8ce service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:05.454950 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce611e-5607-4aba-9378-14a702f5c8ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:05.455229 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abce611e-5607-4aba-9378-14a702f5c8ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:05.465535 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce611e-5607-4aba-9378-14a702f5c8ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:01:05.466985 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abce611e-5607-4aba-9378-14a702f5c8ce service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:01:05.477086 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 717/1443] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:01:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:01:07.824952 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe67dc-f5bd-4d8b-a429-d0017c644674 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:07.825884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe67dc-f5bd-4d8b-a429-d0017c644674 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:07.825884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fbe67dc-f5bd-4d8b-a429-d0017c644674 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:07.828114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 727/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:01:07 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:01:07.833662 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a52c00-62c8-4061-94a0-16ab05fe5229 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:07.834087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a52c00-62c8-4061-94a0-16ab05fe5229 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:07.834282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a52c00-62c8-4061-94a0-16ab05fe5229 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:07.852654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d9a52c00-62c8-4061-94a0-16ab05fe5229 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:01:07.853856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d9a52c00-62c8-4061-94a0-16ab05fe5229 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:01:07.862220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 718/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:01:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:01:07.900170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe67dc-f5bd-4d8b-a429-d0017c644674 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:01:07.913133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fbe67dc-f5bd-4d8b-a429-d0017c644674 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:01:07.917551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff77ae6-ff69-4ac7-856c-355ce403a60e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:01:07.918193 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff77ae6-ff69-4ac7-856c-355ce403a60e service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:07.918488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff77ae6-ff69-4ac7-856c-355ce403a60e service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:07.918765 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff77ae6-ff69-4ac7-856c-355ce403a60e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:07.929251 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff77ae6-ff69-4ac7-856c-355ce403a60e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:01:07.931292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff77ae6-ff69-4ac7-856c-355ce403a60e service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:01:07.942246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 728/1446] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:01:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:01:10.428948 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242f7b8-6a66-4a03-a3cd-f4e0616ad22f None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:10.429507 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242f7b8-6a66-4a03-a3cd-f4e0616ad22f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:10.429507 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3242f7b8-6a66-4a03-a3cd-f4e0616ad22f None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:10.430946 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 719/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:01:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:01:10.436896 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78490ced-7613-4cb5-8f60-b4b2d603ac92 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:10.437370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78490ced-7613-4cb5-8f60-b4b2d603ac92 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:10.437479 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78490ced-7613-4cb5-8f60-b4b2d603ac92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:10.457668 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-78490ced-7613-4cb5-8f60-b4b2d603ac92 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:01:10.458869 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-78490ced-7613-4cb5-8f60-b4b2d603ac92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:01:10.467597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 729/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:01:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:01:10.507156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242f7b8-6a66-4a03-a3cd-f4e0616ad22f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:01:10.519685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3242f7b8-6a66-4a03-a3cd-f4e0616ad22f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:01:10.523732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8252d34-eab4-4369-bf48-3f854d01a436 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:01:10.524402 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8252d34-eab4-4369-bf48-3f854d01a436 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:10.524684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8252d34-eab4-4369-bf48-3f854d01a436 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:10.524962 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8252d34-eab4-4369-bf48-3f854d01a436 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:10.534125 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8252d34-eab4-4369-bf48-3f854d01a436 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:01:10.535550 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8252d34-eab4-4369-bf48-3f854d01a436 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:01:10.545023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 720/1449] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:01:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:01:12.844741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bb29b-64e8-4ad5-a126-6df3c81a7ac0 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:12.845400 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bb29b-64e8-4ad5-a126-6df3c81a7ac0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:12.845400 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8bb29b-64e8-4ad5-a126-6df3c81a7ac0 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:12.846719 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 730/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:01:12.853436 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19c6a1-6e17-474c-a7be-e2d6c7cf3c49 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:12.853889 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19c6a1-6e17-474c-a7be-e2d6c7cf3c49 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:12.853998 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea19c6a1-6e17-474c-a7be-e2d6c7cf3c49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:12.883106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ea19c6a1-6e17-474c-a7be-e2d6c7cf3c49 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:01:12.884438 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ea19c6a1-6e17-474c-a7be-e2d6c7cf3c49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:01:12.893760 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 721/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:01:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:01:12.936641 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bb29b-64e8-4ad5-a126-6df3c81a7ac0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:01:12.952464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8bb29b-64e8-4ad5-a126-6df3c81a7ac0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:01:12.956550 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d81a45-f809-41fe-845e-82553ad472bd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:01:12.957200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d81a45-f809-41fe-845e-82553ad472bd service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:12.957493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d81a45-f809-41fe-845e-82553ad472bd service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:12.957780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d81a45-f809-41fe-845e-82553ad472bd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:12.967481 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d81a45-f809-41fe-845e-82553ad472bd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:01:12.968979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d81a45-f809-41fe-845e-82553ad472bd service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:01:12.978763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 731/1452] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:01:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:01:15.306226 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96380e4-e027-40cb-b5a7-5af3eb2019f1 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:15.306723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96380e4-e027-40cb-b5a7-5af3eb2019f1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:15.306822 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b96380e4-e027-40cb-b5a7-5af3eb2019f1 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:15.308628 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 722/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:01:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:01:15.314105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c86ec-0459-443f-90c5-da41f00b4891 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:15.314562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c86ec-0459-443f-90c5-da41f00b4891 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:15.314664 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8c86ec-0459-443f-90c5-da41f00b4891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:15.333674 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1a8c86ec-0459-443f-90c5-da41f00b4891 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:01:15.334890 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1a8c86ec-0459-443f-90c5-da41f00b4891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:01:15.344234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 732/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:01:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:01:15.388639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96380e4-e027-40cb-b5a7-5af3eb2019f1 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:01:15.404556 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b96380e4-e027-40cb-b5a7-5af3eb2019f1 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:01:15.408751 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b83be64-c146-493a-9cec-0291efab71e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:01:15.409397 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83be64-c146-493a-9cec-0291efab71e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:15.409678 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83be64-c146-493a-9cec-0291efab71e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:15.409956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b83be64-c146-493a-9cec-0291efab71e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:15.419558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83be64-c146-493a-9cec-0291efab71e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:01:15.421040 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b83be64-c146-493a-9cec-0291efab71e3 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:01:15.430386 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 723/1455] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:01:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:01:17.671450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e9f27-6d5b-47d6-ac87-3b9a10088569 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:17.672088 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e9f27-6d5b-47d6-ac87-3b9a10088569 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:17.672088 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-529e9f27-6d5b-47d6-ac87-3b9a10088569 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:17.673451 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 733/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:01:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:01:17.680158 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fe04d-ac47-4669-83ed-02d087b06006 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:17.680601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fe04d-ac47-4669-83ed-02d087b06006 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:17.680705 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7fe04d-ac47-4669-83ed-02d087b06006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:17.699532 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4f7fe04d-ac47-4669-83ed-02d087b06006 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:01:17.700787 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4f7fe04d-ac47-4669-83ed-02d087b06006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:01:17.709526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 724/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:01:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:01:17.746857 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e9f27-6d5b-47d6-ac87-3b9a10088569 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:01:17.759142 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-529e9f27-6d5b-47d6-ac87-3b9a10088569 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:01:17.763600 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc4c49f-68e6-49e5-8940-82b1dd709b84 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:01:17.764269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4c49f-68e6-49e5-8940-82b1dd709b84 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:17.764564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4c49f-68e6-49e5-8940-82b1dd709b84 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:17.764845 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc4c49f-68e6-49e5-8940-82b1dd709b84 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:17.775269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4c49f-68e6-49e5-8940-82b1dd709b84 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:01:17.776784 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc4c49f-68e6-49e5-8940-82b1dd709b84 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:01:17.786850 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 734/1458] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:01:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:01:20.030489 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4adbf-5aab-4df5-ac3e-21ade30c963f None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:20.031109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4adbf-5aab-4df5-ac3e-21ade30c963f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:20.031109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b4adbf-5aab-4df5-ac3e-21ade30c963f None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:20.032463 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 725/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:01:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:01:20.039449 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d14953-5808-44d4-9a39-a20351aff5b4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:20.040199 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d14953-5808-44d4-9a39-a20351aff5b4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:20.040392 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79d14953-5808-44d4-9a39-a20351aff5b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:20.050636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=70 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:01:20.064603 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-79d14953-5808-44d4-9a39-a20351aff5b4 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:01:20.065823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-79d14953-5808-44d4-9a39-a20351aff5b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:01:20.074985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 735/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:01:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:01:20.112559 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4adbf-5aab-4df5-ac3e-21ade30c963f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:01:20.118480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=73 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:01:20.125076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b4adbf-5aab-4df5-ac3e-21ade30c963f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:01:20.128832 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab5b45e-08be-4d1b-a73f-dc4ecf59c797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:01:20.129478 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5b45e-08be-4d1b-a73f-dc4ecf59c797 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:20.129764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5b45e-08be-4d1b-a73f-dc4ecf59c797 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:20.130058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab5b45e-08be-4d1b-a73f-dc4ecf59c797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:20.139302 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5b45e-08be-4d1b-a73f-dc4ecf59c797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:01:20.140787 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ab5b45e-08be-4d1b-a73f-dc4ecf59c797 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:01:20.150575 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 726/1461] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:01:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:01:22.396458 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390782f-b277-45a3-b3e1-853aeeeea62f None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:22.397036 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390782f-b277-45a3-b3e1-853aeeeea62f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:22.397036 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6390782f-b277-45a3-b3e1-853aeeeea62f None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:22.398495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 736/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:01:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:01:22.405173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bc971-1715-47d8-88b3-89fca5053a92 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:22.405614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bc971-1715-47d8-88b3-89fca5053a92 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:22.405774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197bc971-1715-47d8-88b3-89fca5053a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:22.428363 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-197bc971-1715-47d8-88b3-89fca5053a92 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:01:22.429576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-197bc971-1715-47d8-88b3-89fca5053a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:01:22.439362 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 727/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:01:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:01:22.478879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390782f-b277-45a3-b3e1-853aeeeea62f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:01:22.494059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6390782f-b277-45a3-b3e1-853aeeeea62f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:01:22.501281 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b02362-589d-49db-a7eb-7ec71273c7f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:01:22.502245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b02362-589d-49db-a7eb-7ec71273c7f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:22.502678 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b02362-589d-49db-a7eb-7ec71273c7f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:22.503084 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b02362-589d-49db-a7eb-7ec71273c7f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:22.514588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b02362-589d-49db-a7eb-7ec71273c7f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:01:22.516166 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b02362-589d-49db-a7eb-7ec71273c7f1 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:01:22.526063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 737/1464] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:01:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:01:24.725982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bc987-e5e9-4fb1-a425-187f387c90f2 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:24.726743 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bc987-e5e9-4fb1-a425-187f387c90f2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:24.726743 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e13bc987-e5e9-4fb1-a425-187f387c90f2 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:24.728263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 728/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:01:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:01:24.736165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e189-151f-4816-8d42-7f68d2ac7742 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:24.736600 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e189-151f-4816-8d42-7f68d2ac7742 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:24.736757 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8139e189-151f-4816-8d42-7f68d2ac7742 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:24.755220 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8139e189-151f-4816-8d42-7f68d2ac7742 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:01:24.756455 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8139e189-151f-4816-8d42-7f68d2ac7742 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:01:24.765051 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 738/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:01:24.801175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bc987-e5e9-4fb1-a425-187f387c90f2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:01:24.813379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e13bc987-e5e9-4fb1-a425-187f387c90f2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:01:24.817395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e674d-3806-4b20-8833-cffa634c85f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:01:24.818038 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e674d-3806-4b20-8833-cffa634c85f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:24.818325 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e674d-3806-4b20-8833-cffa634c85f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:24.818612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e674d-3806-4b20-8833-cffa634c85f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:24.828082 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e674d-3806-4b20-8833-cffa634c85f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:01:24.829501 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e674d-3806-4b20-8833-cffa634c85f1 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:01:24.840500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 729/1467] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:01:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:01:27.030579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005add7-8878-455a-a39d-3b07687bad5a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:27.031189 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005add7-8878-455a-a39d-3b07687bad5a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:27.031189 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005add7-8878-455a-a39d-3b07687bad5a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:27.032496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 739/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:01:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:01:27.039352 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff9a12-f515-4d57-9905-8773c0df7766 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:27.039794 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff9a12-f515-4d57-9905-8773c0df7766 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:27.039922 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44ff9a12-f515-4d57-9905-8773c0df7766 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:27.068985 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-44ff9a12-f515-4d57-9905-8773c0df7766 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:01:27.070629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-44ff9a12-f515-4d57-9905-8773c0df7766 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:01:27.080151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 730/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:01:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:01:27.116425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005add7-8878-455a-a39d-3b07687bad5a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:01:27.128822 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005add7-8878-455a-a39d-3b07687bad5a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:01:27.133266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8951496a-07ba-4b48-9784-9c181d0de5b8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:01:27.133904 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951496a-07ba-4b48-9784-9c181d0de5b8 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:27.134190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951496a-07ba-4b48-9784-9c181d0de5b8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:27.134457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8951496a-07ba-4b48-9784-9c181d0de5b8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:27.143815 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951496a-07ba-4b48-9784-9c181d0de5b8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:01:27.145284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8951496a-07ba-4b48-9784-9c181d0de5b8 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:01:27.154694 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 740/1470] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:01:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:01:37.060518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=17 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:01:37.150452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=17 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:01:59.710878 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f4ecb-dee9-4803-8993-18c1ace497b4 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:59.711525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f4ecb-dee9-4803-8993-18c1ace497b4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:59.711525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704f4ecb-dee9-4803-8993-18c1ace497b4 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:59.712867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 731/1471] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:01:59.719169 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da6506-8a1c-4b03-b0eb-d7d3b8b52249 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:59.720007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da6506-8a1c-4b03-b0eb-d7d3b8b52249 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:59.720178 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13da6506-8a1c-4b03-b0eb-d7d3b8b52249 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:59.747616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-13da6506-8a1c-4b03-b0eb-d7d3b8b52249 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:01:59.748878 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-13da6506-8a1c-4b03-b0eb-d7d3b8b52249 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:01:59.758110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 741/1472] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:01:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:01:59.810704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46635d99-5322-480d-a91d-ca411626f042 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:59.810994 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46635d99-5322-480d-a91d-ca411626f042 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:59.811270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46635d99-5322-480d-a91d-ca411626f042 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:59.812692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 732/1473] 10.222.0.85 () {54 vars in 962 bytes} [Wed May 25 16:01:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:01:59.817604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2a36e-18d8-4228-ac2b-c95106557cf7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:59.817935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2a36e-18d8-4228-ac2b-c95106557cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:59.818196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a2a36e-18d8-4228-ac2b-c95106557cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:59.838104 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-61a2a36e-18d8-4228-ac2b-c95106557cf7 None None] MFA Rules not processed for user `196dc406a95444c39babc50d59f9eb6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:01:59.839667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-61a2a36e-18d8-4228-ac2b-c95106557cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:01:59.847282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 742/1474] 10.222.0.85 () {58 vars in 1059 bytes} [Wed May 25 16:01:59 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:01:59.854510 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46635d99-5322-480d-a91d-ca411626f042 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:01:59.866877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46635d99-5322-480d-a91d-ca411626f042 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:01:59.871115 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a889210-4239-4707-979d-9cba39d4f300 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '196dc406a95444c39babc50d59f9eb6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:01:59.871757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a889210-4239-4707-979d-9cba39d4f300 service glance] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:01:59.872081 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a889210-4239-4707-979d-9cba39d4f300 service glance] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:01:59.872372 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a889210-4239-4707-979d-9cba39d4f300 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:01:59.881859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a889210-4239-4707-979d-9cba39d4f300 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:01:59.883320 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a889210-4239-4707-979d-9cba39d4f300 service glance] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:01:59.892961 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 733/1475] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:01:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:02:02.187167 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabf43d-bc81-489e-ab57-4fe421f474e4 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:02.188099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabf43d-bc81-489e-ab57-4fe421f474e4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:02.188099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daabf43d-bc81-489e-ab57-4fe421f474e4 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:02.190404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 743/1476] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:02:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:02:02.196021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15082b2c-4b68-44a8-8e13-159163e38c4b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:02.196504 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15082b2c-4b68-44a8-8e13-159163e38c4b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:02.196610 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15082b2c-4b68-44a8-8e13-159163e38c4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:02.214612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-15082b2c-4b68-44a8-8e13-159163e38c4b None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:02:02.215800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-15082b2c-4b68-44a8-8e13-159163e38c4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:02:02.224271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 734/1477] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:02:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:02:02.247789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabf43d-bc81-489e-ab57-4fe421f474e4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:02:02.265851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daabf43d-bc81-489e-ab57-4fe421f474e4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:02:02.271849 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a97e7b-df85-4d8b-a8a2-3f42ed88c586 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:02:02.272493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a97e7b-df85-4d8b-a8a2-3f42ed88c586 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:02.272774 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a97e7b-df85-4d8b-a8a2-3f42ed88c586 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:02.273053 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a97e7b-df85-4d8b-a8a2-3f42ed88c586 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:02.273787 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a97e7b-df85-4d8b-a8a2-3f42ed88c586 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:02:02.275633 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-96a97e7b-df85-4d8b-a8a2-3f42ed88c586 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:02:02.275633 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 25 16:02:02.275633 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: May 25 16:02:02.276786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a97e7b-df85-4d8b-a8a2-3f42ed88c586 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:02:02.297761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 744/1478] 10.222.0.85 () {56 vars in 1108 bytes} [Wed May 25 16:02:02 2022] GET /identity/v3/projects => generated 2553 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:02:04.505773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c51e9c-b742-479d-be69-3f515e70ca4c None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:04.506340 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c51e9c-b742-479d-be69-3f515e70ca4c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:04.506340 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3c51e9c-b742-479d-be69-3f515e70ca4c None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:04.508448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 735/1479] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:02:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:02:04.517925 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e068666-9ee6-48c3-b042-575dd025d635 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:04.518412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e068666-9ee6-48c3-b042-575dd025d635 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:04.518522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e068666-9ee6-48c3-b042-575dd025d635 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:04.539772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2e068666-9ee6-48c3-b042-575dd025d635 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:02:04.541021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2e068666-9ee6-48c3-b042-575dd025d635 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:02:04.550614 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 745/1480] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:02:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:02:04.593607 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c51e9c-b742-479d-be69-3f515e70ca4c None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:02:04.609038 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3c51e9c-b742-479d-be69-3f515e70ca4c None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:02:04.615181 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9735188d-8f60-4405-9dcc-cabbf082c16d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:02:04.615858 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735188d-8f60-4405-9dcc-cabbf082c16d service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:04.616138 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735188d-8f60-4405-9dcc-cabbf082c16d service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:04.616418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9735188d-8f60-4405-9dcc-cabbf082c16d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:04.625291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735188d-8f60-4405-9dcc-cabbf082c16d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:02:04.627091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9735188d-8f60-4405-9dcc-cabbf082c16d service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:02:04.638771 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 736/1481] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:02:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:02:06.986508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac417f-7aca-4712-a226-331a309c6c16 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:06.987020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac417f-7aca-4712-a226-331a309c6c16 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:06.987115 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfac417f-7aca-4712-a226-331a309c6c16 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:06.988495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 746/1482] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:02:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:02:06.994118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f774222-a659-4722-b162-b89e2088d834 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:06.994525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f774222-a659-4722-b162-b89e2088d834 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:06.994718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f774222-a659-4722-b162-b89e2088d834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:07.013783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6f774222-a659-4722-b162-b89e2088d834 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:02:07.014996 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6f774222-a659-4722-b162-b89e2088d834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:02:07.023635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 737/1483] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:02:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:02:07.062686 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac417f-7aca-4712-a226-331a309c6c16 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:02:07.075585 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfac417f-7aca-4712-a226-331a309c6c16 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:02:07.079668 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eb5bf5a-5cb8-48f4-8b3b-76489b1b6645 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:02:07.080345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5bf5a-5cb8-48f4-8b3b-76489b1b6645 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:07.080638 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5bf5a-5cb8-48f4-8b3b-76489b1b6645 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:07.080919 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb5bf5a-5cb8-48f4-8b3b-76489b1b6645 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:07.090302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5bf5a-5cb8-48f4-8b3b-76489b1b6645 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:02:07.091805 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eb5bf5a-5cb8-48f4-8b3b-76489b1b6645 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:02:07.101665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 747/1484] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:02:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:02:09.328072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6819d-8119-4e69-bfee-02da79461385 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:09.328595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6819d-8119-4e69-bfee-02da79461385 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:09.328595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b6819d-8119-4e69-bfee-02da79461385 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:09.330067 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 738/1485] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:02:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:02:09.335442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498985f-ff0d-4664-bc4c-98f508be2eed None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:09.335891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498985f-ff0d-4664-bc4c-98f508be2eed None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:09.336019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9498985f-ff0d-4664-bc4c-98f508be2eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:09.354726 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9498985f-ff0d-4664-bc4c-98f508be2eed None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:02:09.355976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9498985f-ff0d-4664-bc4c-98f508be2eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:02:09.366055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 748/1486] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:02:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:02:09.409519 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6819d-8119-4e69-bfee-02da79461385 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:02:09.429328 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b6819d-8119-4e69-bfee-02da79461385 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:02:09.433684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34691494-cea9-4698-8b54-70f3e22ef632 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:02:09.434334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34691494-cea9-4698-8b54-70f3e22ef632 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:09.434619 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34691494-cea9-4698-8b54-70f3e22ef632 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:09.434899 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34691494-cea9-4698-8b54-70f3e22ef632 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:09.444428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34691494-cea9-4698-8b54-70f3e22ef632 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:02:09.445865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34691494-cea9-4698-8b54-70f3e22ef632 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:02:09.455616 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 739/1487] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:02:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:02:11.763723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c7e71-5b94-4ae4-9438-ce29347b3d2d None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:11.764370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c7e71-5b94-4ae4-9438-ce29347b3d2d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:11.764370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41c7e71-5b94-4ae4-9438-ce29347b3d2d None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:11.765766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 749/1488] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:02:11.772450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a82b9e-930c-4a74-aed1-a6c58b396171 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:11.772871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a82b9e-930c-4a74-aed1-a6c58b396171 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:11.772981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a82b9e-930c-4a74-aed1-a6c58b396171 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:11.802181 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-73a82b9e-930c-4a74-aed1-a6c58b396171 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:02:11.803729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-73a82b9e-930c-4a74-aed1-a6c58b396171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:02:11.812487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 740/1489] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:02:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:02:11.852226 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c7e71-5b94-4ae4-9438-ce29347b3d2d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:02:11.865258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41c7e71-5b94-4ae4-9438-ce29347b3d2d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:02:11.869195 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a9435f-0c70-4392-8907-f653e2be31bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:02:11.869845 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9435f-0c70-4392-8907-f653e2be31bf service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:11.870137 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9435f-0c70-4392-8907-f653e2be31bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:11.870418 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a9435f-0c70-4392-8907-f653e2be31bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:11.879564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9435f-0c70-4392-8907-f653e2be31bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:02:11.881062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a9435f-0c70-4392-8907-f653e2be31bf service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:02:11.890683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 750/1490] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:02:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:02:14.270582 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550509d-7b47-44e7-8542-3f82f026bac0 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:14.271175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550509d-7b47-44e7-8542-3f82f026bac0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:14.271175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5550509d-7b47-44e7-8542-3f82f026bac0 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:14.273343 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 741/1491] 10.222.0.85 () {54 vars in 926 bytes} [Wed May 25 16:02:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:02:14.278725 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8b3a5-e61a-434c-82aa-8e59e37efa25 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:14.279172 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8b3a5-e61a-434c-82aa-8e59e37efa25 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:14.279287 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bf8b3a5-e61a-434c-82aa-8e59e37efa25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:14.297812 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6bf8b3a5-e61a-434c-82aa-8e59e37efa25 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:02:14.299032 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6bf8b3a5-e61a-434c-82aa-8e59e37efa25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:02:14.307460 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 751/1492] 10.222.0.85 () {58 vars in 1023 bytes} [Wed May 25 16:02:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:02:14.346250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550509d-7b47-44e7-8542-3f82f026bac0 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:02:14.358175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5550509d-7b47-44e7-8542-3f82f026bac0 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:02:14.361921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56223bb2-6171-4ba2-8be9-5e8e4649c078 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:02:14.362567 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56223bb2-6171-4ba2-8be9-5e8e4649c078 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:02:14.362855 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56223bb2-6171-4ba2-8be9-5e8e4649c078 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:02:14.363136 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56223bb2-6171-4ba2-8be9-5e8e4649c078 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:02:14.372219 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56223bb2-6171-4ba2-8be9-5e8e4649c078 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:02:14.373665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56223bb2-6171-4ba2-8be9-5e8e4649c078 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:02:14.383203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 742/1493] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:02:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:02:24.292703 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=33 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:02:24.379230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=33 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:03:58.613010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adacbe-8ee9-4542-a456-352ff800ef3e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:03:58.613937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adacbe-8ee9-4542-a456-352ff800ef3e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:03:58.613937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90adacbe-8ee9-4542-a456-352ff800ef3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:03:58.634652 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-90adacbe-8ee9-4542-a456-352ff800ef3e None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:03:58.635871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-90adacbe-8ee9-4542-a456-352ff800ef3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:03:58.645103 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 752/1494] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:03:58.654318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a006a6e-f541-48b0-8361-c791bfbfc508 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:03:58.654855 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a006a6e-f541-48b0-8361-c791bfbfc508 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:03:58.654855 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a006a6e-f541-48b0-8361-c791bfbfc508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:03:58.673645 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6a006a6e-f541-48b0-8361-c791bfbfc508 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:03:58.674827 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6a006a6e-f541-48b0-8361-c791bfbfc508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:03:58.683621 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 743/1495] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:03:58.688254 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adacbe-8ee9-4542-a456-352ff800ef3e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:03:58.700101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90adacbe-8ee9-4542-a456-352ff800ef3e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:03:58.707345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed82361-e1a5-4ff7-8701-17873b89fbee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:03:58.708158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed82361-e1a5-4ff7-8701-17873b89fbee admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:03:58.708411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed82361-e1a5-4ff7-8701-17873b89fbee admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:03:58.708725 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed82361-e1a5-4ff7-8701-17873b89fbee admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:03:58.709498 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed82361-e1a5-4ff7-8701-17873b89fbee admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:03:58.709813 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed82361-e1a5-4ff7-8701-17873b89fbee admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:03:58.710835 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-4ed82361-e1a5-4ff7-8701-17873b89fbee admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 16:03:58.710835 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 25 16:03:58.710835 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: May 25 16:03:58.711761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed82361-e1a5-4ff7-8701-17873b89fbee admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:03:58.725995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 753/1496] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:03:58.733667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a006a6e-f541-48b0-8361-c791bfbfc508 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:03:58.743461 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a006a6e-f541-48b0-8361-c791bfbfc508 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:03:58.748109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d869396-d4a5-4bbc-8e84-8a0532049250 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:03:58.748768 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d869396-d4a5-4bbc-8e84-8a0532049250 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:03:58.749055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d869396-d4a5-4bbc-8e84-8a0532049250 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:03:58.749331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d869396-d4a5-4bbc-8e84-8a0532049250 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:03:58.750019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d869396-d4a5-4bbc-8e84-8a0532049250 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:03:58.751807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: WARNING py.warnings [None req-8d869396-d4a5-4bbc-8e84-8a0532049250 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:03:58.751807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: warnings.warn(msg) May 25 16:03:58.751807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: May 25 16:03:58.753027 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d869396-d4a5-4bbc-8e84-8a0532049250 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:03:58.790095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 744/1497] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:03:58 2022] POST /identity/v3/projects => generated 377 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:03:58.794389 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed82361-e1a5-4ff7-8701-17873b89fbee admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:03:58.804987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed82361-e1a5-4ff7-8701-17873b89fbee admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:03:58.864562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42291617-2758-45a7-b2eb-5d31d5d3cde8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:03:58.865176 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42291617-2758-45a7-b2eb-5d31d5d3cde8 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:03:58.865467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42291617-2758-45a7-b2eb-5d31d5d3cde8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:03:58.865744 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42291617-2758-45a7-b2eb-5d31d5d3cde8 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:03:58.866486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42291617-2758-45a7-b2eb-5d31d5d3cde8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:03:58.867525 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-42291617-2758-45a7-b2eb-5d31d5d3cde8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:03:58.867525 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 25 16:03:58.867525 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: May 25 16:03:58.868766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42291617-2758-45a7-b2eb-5d31d5d3cde8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:03:58.890641 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 754/1498] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:03:58 2022] POST /identity/v3/users => generated 289 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:03:58.894868 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d869396-d4a5-4bbc-8e84-8a0532049250 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:03:58.905098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d869396-d4a5-4bbc-8e84-8a0532049250 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:03:58.909808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33785cfd-12f2-4e21-8ad7-04264fef0c0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:03:58.910472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33785cfd-12f2-4e21-8ad7-04264fef0c0c admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:03:58.910779 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33785cfd-12f2-4e21-8ad7-04264fef0c0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:03:58.911056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33785cfd-12f2-4e21-8ad7-04264fef0c0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:03:58.911802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33785cfd-12f2-4e21-8ad7-04264fef0c0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:03:58.912842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: WARNING py.warnings [None req-33785cfd-12f2-4e21-8ad7-04264fef0c0c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 16:03:58.912842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: warnings.warn(msg) May 25 16:03:58.912842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: May 25 16:03:58.913726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33785cfd-12f2-4e21-8ad7-04264fef0c0c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:03:58.927049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 745/1499] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:03:58.931131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42291617-2758-45a7-b2eb-5d31d5d3cde8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:03:58.941129 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42291617-2758-45a7-b2eb-5d31d5d3cde8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:03:58.946202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-687eb55a-c9ef-473e-b599-76125ac18175 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:03:58.946856 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687eb55a-c9ef-473e-b599-76125ac18175 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:03:58.947139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687eb55a-c9ef-473e-b599-76125ac18175 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:03:58.947419 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-687eb55a-c9ef-473e-b599-76125ac18175 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:03:58.948178 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687eb55a-c9ef-473e-b599-76125ac18175 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:03:58.949202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-687eb55a-c9ef-473e-b599-76125ac18175 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 16:03:58.949202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 25 16:03:58.949202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: May 25 16:03:58.950075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-687eb55a-c9ef-473e-b599-76125ac18175 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:03:58.963709 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 755/1500] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:03:58.967788 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33785cfd-12f2-4e21-8ad7-04264fef0c0c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:03:58.977808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33785cfd-12f2-4e21-8ad7-04264fef0c0c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:03:58.982954 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073f318e-265b-4669-b15b-6cb7a80c1499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:03:58.983848 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f318e-265b-4669-b15b-6cb7a80c1499 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:03:58.984164 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f318e-265b-4669-b15b-6cb7a80c1499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:03:58.984430 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073f318e-265b-4669-b15b-6cb7a80c1499 admin admin] PATH_INFO: `/v3/projects/edad283147b048a791a625bbbe1a11b0/users/02e20b6447154f0c8393cc3c7f4077f2/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:03:58.999063 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f318e-265b-4669-b15b-6cb7a80c1499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=edad283147b048a791a625bbbe1a11b0, user_id=02e20b6447154f0c8393cc3c7f4077f2, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:03:59.000143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: WARNING py.warnings [None req-073f318e-265b-4669-b15b-6cb7a80c1499 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:03:59.000143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: warnings.warn(msg) May 25 16:03:59.000143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: May 25 16:03:59.002061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073f318e-265b-4669-b15b-6cb7a80c1499 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:03:59.016248 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 746/1501] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:03:58 2022] PUT /identity/v3/projects/edad283147b048a791a625bbbe1a11b0/users/02e20b6447154f0c8393cc3c7f4077f2/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:03:59.028038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba8f81-45dc-4fa0-9bf7-8c96a28398ab None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:03:59.028337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba8f81-45dc-4fa0-9bf7-8c96a28398ab None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:03:59.028619 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fba8f81-45dc-4fa0-9bf7-8c96a28398ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:03:59.056807 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9fba8f81-45dc-4fa0-9bf7-8c96a28398ab None None] MFA Rules not processed for user `02e20b6447154f0c8393cc3c7f4077f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:03:59.058370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9fba8f81-45dc-4fa0-9bf7-8c96a28398ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:03:59.134229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 756/1502] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:03:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:03:59.147814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4680e74-dc5b-45ba-8d83-22c1c5ea2f42 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:03:59.148154 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4680e74-dc5b-45ba-8d83-22c1c5ea2f42 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:03:59.148437 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4680e74-dc5b-45ba-8d83-22c1c5ea2f42 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:03:59.149870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 747/1503] 10.222.0.85 () {54 vars in 963 bytes} [Wed May 25 16:03:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:03:59.154765 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9058025-9729-4dda-bed2-8b406d827702 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:03:59.155066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9058025-9729-4dda-bed2-8b406d827702 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:03:59.155351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9058025-9729-4dda-bed2-8b406d827702 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:03:59.174116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c9058025-9729-4dda-bed2-8b406d827702 None None] MFA Rules not processed for user `f81aecdc87d84501ba94b208405b51dd`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:03:59.175306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c9058025-9729-4dda-bed2-8b406d827702 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:03:59.223385 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:03:59.237393 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 757/1504] 10.222.0.85 () {58 vars in 1060 bytes} [Wed May 25 16:03:59 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:03:59.249587 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4680e74-dc5b-45ba-8d83-22c1c5ea2f42 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:03:59.262504 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4680e74-dc5b-45ba-8d83-22c1c5ea2f42 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:03:59.266754 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47beb2d-c4fa-4bb3-b418-60b3926b4faa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:03:59.267400 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47beb2d-c4fa-4bb3-b418-60b3926b4faa service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:03:59.267693 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47beb2d-c4fa-4bb3-b418-60b3926b4faa service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:03:59.268025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47beb2d-c4fa-4bb3-b418-60b3926b4faa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:03:59.276876 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47beb2d-c4fa-4bb3-b418-60b3926b4faa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:03:59.278329 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47beb2d-c4fa-4bb3-b418-60b3926b4faa service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:03:59.287278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 748/1505] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:03:59 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:03:59.331462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcc182-a463-4747-8271-0a1c5190e3af None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:03:59.331758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcc182-a463-4747-8271-0a1c5190e3af None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:03:59.332079 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fcc182-a463-4747-8271-0a1c5190e3af None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:03:59.333420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 758/1506] 10.222.0.85 () {52 vars in 829 bytes} [Wed May 25 16:03:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:03:59.337895 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47beb2d-c4fa-4bb3-b418-60b3926b4faa service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:03:59.426895 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47beb2d-c4fa-4bb3-b418-60b3926b4faa service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:03:59.432176 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121117c-290d-4912-b46f-33415a275bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:03:59.433066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121117c-290d-4912-b46f-33415a275bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:03:59.433352 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121117c-290d-4912-b46f-33415a275bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:03:59.433622 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0121117c-290d-4912-b46f-33415a275bab admin admin] PATH_INFO: `/v3/users/02e20b6447154f0c8393cc3c7f4077f2` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:03:59.434868 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0121117c-290d-4912-b46f-33415a275bab admin admin] RBAC: Authorizing `identity:delete_user(user_id=02e20b6447154f0c8393cc3c7f4077f2)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:03:59.435929 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: WARNING py.warnings [None req-0121117c-290d-4912-b46f-33415a275bab admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:03:59.435929 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: warnings.warn(msg) May 25 16:03:59.435929 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: May 25 16:03:59.437130 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0121117c-290d-4912-b46f-33415a275bab admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:03:59.508885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0121117c-290d-4912-b46f-33415a275bab admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02e20b6447154f0c8393cc3c7f4077f2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:03:59.509786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0121117c-290d-4912-b46f-33415a275bab admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02e20b6447154f0c8393cc3c7f4077f2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:03:59.529905 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0121117c-290d-4912-b46f-33415a275bab admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02e20b6447154f0c8393cc3c7f4077f2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:03:59.542008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0121117c-290d-4912-b46f-33415a275bab admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02e20b6447154f0c8393cc3c7f4077f2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:03:59.542665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0121117c-290d-4912-b46f-33415a275bab admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02e20b6447154f0c8393cc3c7f4077f2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:03:59.551953 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:03:59.564058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 749/1507] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:03:59 2022] DELETE /identity/v3/users/02e20b6447154f0c8393cc3c7f4077f2 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:03:59.568654 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c76283-398c-4955-927a-4669fb7e28f0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:03:59.568955 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c76283-398c-4955-927a-4669fb7e28f0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:03:59.569242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45c76283-398c-4955-927a-4669fb7e28f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:03:59.587299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-45c76283-398c-4955-927a-4669fb7e28f0 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:03:59.588566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-45c76283-398c-4955-927a-4669fb7e28f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:03:59.650588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 759/1508] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:03:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:03:59.660453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121117c-290d-4912-b46f-33415a275bab admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:03:59.662233 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0121117c-290d-4912-b46f-33415a275bab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:03:59.725228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0121117c-290d-4912-b46f-33415a275bab admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:03:59.730884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c843fd-1b1a-4a92-a9b7-0262620851c9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:03:59.731895 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c843fd-1b1a-4a92-a9b7-0262620851c9 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:03:59.732321 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c843fd-1b1a-4a92-a9b7-0262620851c9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:03:59.732712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c843fd-1b1a-4a92-a9b7-0262620851c9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:03:59.743835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c843fd-1b1a-4a92-a9b7-0262620851c9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:03:59.745309 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c843fd-1b1a-4a92-a9b7-0262620851c9 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:03:59.754738 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 750/1509] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:03:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:04:00.147130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c76283-398c-4955-927a-4669fb7e28f0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:04:00.148630 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-45c76283-398c-4955-927a-4669fb7e28f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:04:00.161276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45c76283-398c-4955-927a-4669fb7e28f0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:04:00.166960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007d8d2e-112d-4468-9eb7-fd6fd431eb43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:04:00.167875 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d8d2e-112d-4468-9eb7-fd6fd431eb43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:04:00.168208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d8d2e-112d-4468-9eb7-fd6fd431eb43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:04:00.168475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007d8d2e-112d-4468-9eb7-fd6fd431eb43 admin admin] PATH_INFO: `/v3/projects/edad283147b048a791a625bbbe1a11b0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:04:00.169801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d8d2e-112d-4468-9eb7-fd6fd431eb43 admin admin] RBAC: Authorizing `identity:delete_project(project_id=edad283147b048a791a625bbbe1a11b0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:04:00.170866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-007d8d2e-112d-4468-9eb7-fd6fd431eb43 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:04:00.170866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 25 16:04:00.170866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: May 25 16:04:00.172480 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007d8d2e-112d-4468-9eb7-fd6fd431eb43 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:04:00.246943 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-007d8d2e-112d-4468-9eb7-fd6fd431eb43 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'edad283147b048a791a625bbbe1a11b0', 'request_id': 'req-007d8d2e-112d-4468-9eb7-fd6fd431eb43'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:04:00.259064 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-007d8d2e-112d-4468-9eb7-fd6fd431eb43 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'edad283147b048a791a625bbbe1a11b0', 'request_id': 'req-007d8d2e-112d-4468-9eb7-fd6fd431eb43'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:04:00.273427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-007d8d2e-112d-4468-9eb7-fd6fd431eb43 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'edad283147b048a791a625bbbe1a11b0', 'request_id': 'req-007d8d2e-112d-4468-9eb7-fd6fd431eb43'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:04:00.286312 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-007d8d2e-112d-4468-9eb7-fd6fd431eb43 admin admin] The token cache is being invalidate because project edad283147b048a791a625bbbe1a11b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:04:00.286608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-007d8d2e-112d-4468-9eb7-fd6fd431eb43 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:04:00.287375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-007d8d2e-112d-4468-9eb7-fd6fd431eb43 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:04:00.289231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 760/1510] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:04:00 2022] DELETE /identity/v3/projects/edad283147b048a791a625bbbe1a11b0 => generated 0 bytes in 143 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:04:09.750547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=29 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:04:10.282837 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:05:43.662237 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6c6e7-8250-4151-9042-8659402384a5 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:43.662850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6c6e7-8250-4151-9042-8659402384a5 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:43.662850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae6c6e7-8250-4151-9042-8659402384a5 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:43.665424 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 751/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:05:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:43.671091 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3d2c0-ab43-4594-9297-5fe47fcc159f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:43.671494 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3d2c0-ab43-4594-9297-5fe47fcc159f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:43.671662 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3d2c0-ab43-4594-9297-5fe47fcc159f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:43.690768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-38d3d2c0-ab43-4594-9297-5fe47fcc159f None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:43.692012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-38d3d2c0-ab43-4594-9297-5fe47fcc159f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:43.762453 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 761/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:05:43.934428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6c6e7-8250-4151-9042-8659402384a5 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:43.936498 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2ae6c6e7-8250-4151-9042-8659402384a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:44.013007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae6c6e7-8250-4151-9042-8659402384a5 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:44.019028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb3a60f-fd4c-460f-b22d-84f7822acf1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:44.019692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3a60f-fd4c-460f-b22d-84f7822acf1b service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:44.020017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3a60f-fd4c-460f-b22d-84f7822acf1b service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:44.020303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb3a60f-fd4c-460f-b22d-84f7822acf1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:44.028918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3a60f-fd4c-460f-b22d-84f7822acf1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:44.030553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb3a60f-fd4c-460f-b22d-84f7822acf1b service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:44.041449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 752/1513] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:05:44.908666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5379591-1c7a-4901-a790-99bb86539d07 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:44.909444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5379591-1c7a-4901-a790-99bb86539d07 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:44.909612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5379591-1c7a-4901-a790-99bb86539d07 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:44.911882 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 762/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:05:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:44.917106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca2e41-caba-4517-b469-1e707ba70f65 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:44.917605 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca2e41-caba-4517-b469-1e707ba70f65 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:44.917710 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ca2e41-caba-4517-b469-1e707ba70f65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:44.938355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-13ca2e41-caba-4517-b469-1e707ba70f65 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:44.939573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-13ca2e41-caba-4517-b469-1e707ba70f65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:44.948581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 753/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:05:45.095529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5379591-1c7a-4901-a790-99bb86539d07 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:45.116422 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5379591-1c7a-4901-a790-99bb86539d07 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:45.122524 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56662c7a-7b42-4ab7-875e-5a40b097c7b0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:45.123177 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56662c7a-7b42-4ab7-875e-5a40b097c7b0 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:45.123466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56662c7a-7b42-4ab7-875e-5a40b097c7b0 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:45.123749 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56662c7a-7b42-4ab7-875e-5a40b097c7b0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:45.132752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56662c7a-7b42-4ab7-875e-5a40b097c7b0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:45.134420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56662c7a-7b42-4ab7-875e-5a40b097c7b0 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:45.144530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 763/1516] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:05:45.307069 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efbc97-7aa7-4ce5-9417-a77ecd054912 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:45.307454 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efbc97-7aa7-4ce5-9417-a77ecd054912 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:45.307743 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6efbc97-7aa7-4ce5-9417-a77ecd054912 None None] PATH_INFO: `/v3` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:45.309188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 754/1517] 10.222.0.67 () {54 vars in 925 bytes} [Wed May 25 16:05:45 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:05:45.316166 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeace46-40d7-45ec-956d-51ffc56b0aff None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:45.316468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeace46-40d7-45ec-956d-51ffc56b0aff None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:45.316757 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eeace46-40d7-45ec-956d-51ffc56b0aff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:45.338071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6eeace46-40d7-45ec-956d-51ffc56b0aff None None] MFA Rules not processed for user `82b738f1b8c64590955a640c461b4ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:45.339314 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6eeace46-40d7-45ec-956d-51ffc56b0aff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:45.406502 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 764/1518] 10.222.0.67 () {58 vars in 1015 bytes} [Wed May 25 16:05:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:05:49.494878 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63a76f-778f-492c-a08a-3fc13bef6d4a None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:49.495416 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63a76f-778f-492c-a08a-3fc13bef6d4a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:49.495416 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da63a76f-778f-492c-a08a-3fc13bef6d4a None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:49.497189 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 755/1519] 10.222.0.85 () {54 vars in 960 bytes} [Wed May 25 16:05:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:49.502505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e9a64-15cd-4f1e-96c9-39dbe039dca6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:49.503021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e9a64-15cd-4f1e-96c9-39dbe039dca6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:49.503131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e9a64-15cd-4f1e-96c9-39dbe039dca6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:49.521732 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-801e9a64-15cd-4f1e-96c9-39dbe039dca6 None None] MFA Rules not processed for user `82b738f1b8c64590955a640c461b4ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:49.522949 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-801e9a64-15cd-4f1e-96c9-39dbe039dca6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:49.531704 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 765/1520] 10.222.0.85 () {58 vars in 1057 bytes} [Wed May 25 16:05:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:05:49.539993 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63a76f-778f-492c-a08a-3fc13bef6d4a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:49.556279 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da63a76f-778f-492c-a08a-3fc13bef6d4a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:49.564379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf35d23-3c3c-4ca4-9ce9-0f5b020de7a6 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b738f1b8c64590955a640c461b4ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:49.565391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf35d23-3c3c-4ca4-9ce9-0f5b020de7a6 service placement] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:49.565786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf35d23-3c3c-4ca4-9ce9-0f5b020de7a6 service placement] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:49.566124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf35d23-3c3c-4ca4-9ce9-0f5b020de7a6 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:49.577518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf35d23-3c3c-4ca4-9ce9-0f5b020de7a6 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:49.579120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf35d23-3c3c-4ca4-9ce9-0f5b020de7a6 service placement] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:49.588910 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 756/1521] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:05:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:05:56.392688 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ab2ad-0722-427d-9696-9464554b167c None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:56.393510 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ab2ad-0722-427d-9696-9464554b167c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:56.393639 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-491ab2ad-0722-427d-9696-9464554b167c None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:56.395998 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 766/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:05:56 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:56.401840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb04604-48ea-4e50-82dc-372117f15501 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:56.402298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb04604-48ea-4e50-82dc-372117f15501 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:56.402601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb04604-48ea-4e50-82dc-372117f15501 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:56.556394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-2eb04604-48ea-4e50-82dc-372117f15501 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:56.557535 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2eb04604-48ea-4e50-82dc-372117f15501 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:56.566916 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 757/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:05:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:05:56.722435 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ab2ad-0722-427d-9696-9464554b167c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:56.746981 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-491ab2ad-0722-427d-9696-9464554b167c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:56.752872 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2918fe-4b7e-494a-840a-026eef9da9fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:56.753524 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2918fe-4b7e-494a-840a-026eef9da9fc service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:56.753811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2918fe-4b7e-494a-840a-026eef9da9fc service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:56.754094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2918fe-4b7e-494a-840a-026eef9da9fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:56.763189 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2918fe-4b7e-494a-840a-026eef9da9fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:56.764875 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2918fe-4b7e-494a-840a-026eef9da9fc service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:56.775022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 767/1524] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:05:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:05:57.625465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64afcf-60cb-4371-8e44-1952d4d79cf2 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:57.626348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64afcf-60cb-4371-8e44-1952d4d79cf2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:57.626348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef64afcf-60cb-4371-8e44-1952d4d79cf2 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:57.628674 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 758/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed May 25 16:05:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:05:57.636577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1541d4-e030-4537-ba10-e965330c0186 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:57.637070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1541d4-e030-4537-ba10-e965330c0186 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:57.637255 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1541d4-e030-4537-ba10-e965330c0186 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:57.656898 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-cb1541d4-e030-4537-ba10-e965330c0186 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:05:57.658108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cb1541d4-e030-4537-ba10-e965330c0186 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:05:57.667573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 768/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed May 25 16:05:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:05:57.826162 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64afcf-60cb-4371-8e44-1952d4d79cf2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:05:57.847113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef64afcf-60cb-4371-8e44-1952d4d79cf2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:05:57.852978 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52a9a52f-f2b3-4d5d-938a-03b89dd9c4c7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:05:57.853619 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9a52f-f2b3-4d5d-938a-03b89dd9c4c7 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:05:57.853905 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9a52f-f2b3-4d5d-938a-03b89dd9c4c7 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:05:57.854185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52a9a52f-f2b3-4d5d-938a-03b89dd9c4c7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:05:57.862555 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9a52f-f2b3-4d5d-938a-03b89dd9c4c7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:05:57.864154 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52a9a52f-f2b3-4d5d-938a-03b89dd9c4c7 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:05:57.873471 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 759/1527] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:05:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:06:01.888326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e81707-0c48-4a31-b3f5-d3aa858d30bd None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:01.889231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e81707-0c48-4a31-b3f5-d3aa858d30bd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:01.889231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5e81707-0c48-4a31-b3f5-d3aa858d30bd None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:01.891484 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 769/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:06:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:01.897871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2517ade-da16-4a12-9e4e-8166e1fc55ea None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:01.898538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2517ade-da16-4a12-9e4e-8166e1fc55ea None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:01.898806 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2517ade-da16-4a12-9e4e-8166e1fc55ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:01.919575 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f2517ade-da16-4a12-9e4e-8166e1fc55ea None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:01.920801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f2517ade-da16-4a12-9e4e-8166e1fc55ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:01.930534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 760/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:06:01.970840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e81707-0c48-4a31-b3f5-d3aa858d30bd None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:01.973796 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c5e81707-0c48-4a31-b3f5-d3aa858d30bd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:02.048589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5e81707-0c48-4a31-b3f5-d3aa858d30bd None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:02.052763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1011e9-44ec-4e73-b48a-09a7eacf3747 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:02.053420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1011e9-44ec-4e73-b48a-09a7eacf3747 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:02.053707 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1011e9-44ec-4e73-b48a-09a7eacf3747 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:02.053999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1011e9-44ec-4e73-b48a-09a7eacf3747 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:02.063478 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1011e9-44ec-4e73-b48a-09a7eacf3747 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:02.064987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1011e9-44ec-4e73-b48a-09a7eacf3747 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:02.070021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:02.075413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 770/1530] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:06:04.238794 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a138562-0e79-4f67-b033-9591d07bd220 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:04.239370 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a138562-0e79-4f67-b033-9591d07bd220 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:04.239370 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a138562-0e79-4f67-b033-9591d07bd220 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:04.240681 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 761/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:04.246590 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2879d-e1b3-4acf-9994-efaba5cf549b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:04.250775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2879d-e1b3-4acf-9994-efaba5cf549b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:04.250775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25f2879d-e1b3-4acf-9994-efaba5cf549b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:04.267088 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-25f2879d-e1b3-4acf-9994-efaba5cf549b None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:04.268642 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-25f2879d-e1b3-4acf-9994-efaba5cf549b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:04.277969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 771/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:06:04.317104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a138562-0e79-4f67-b033-9591d07bd220 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:04.328553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a138562-0e79-4f67-b033-9591d07bd220 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:04.332274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ecbd7c-f3f6-41cb-975f-f432f7e9d1bc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:04.332919 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecbd7c-f3f6-41cb-975f-f432f7e9d1bc service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:04.333205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecbd7c-f3f6-41cb-975f-f432f7e9d1bc service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:04.333481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ecbd7c-f3f6-41cb-975f-f432f7e9d1bc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:04.343003 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ecbd7c-f3f6-41cb-975f-f432f7e9d1bc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:04.344517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ecbd7c-f3f6-41cb-975f-f432f7e9d1bc service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:04.353814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 762/1533] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:06:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:06:08.425306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31961b-e437-4d5e-b60c-88ebb03cdac1 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:08.425893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31961b-e437-4d5e-b60c-88ebb03cdac1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:08.425893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d31961b-e437-4d5e-b60c-88ebb03cdac1 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:08.427239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 772/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:06:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:08.432833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbed61-31cf-4638-8686-292abec2de53 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:08.433313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbed61-31cf-4638-8686-292abec2de53 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:08.433427 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdbed61-31cf-4638-8686-292abec2de53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:08.453407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8cdbed61-31cf-4638-8686-292abec2de53 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:08.454602 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8cdbed61-31cf-4638-8686-292abec2de53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:08.463722 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 763/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:06:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:06:08.735780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31961b-e437-4d5e-b60c-88ebb03cdac1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:08.738408 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5d31961b-e437-4d5e-b60c-88ebb03cdac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:08.800069 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d31961b-e437-4d5e-b60c-88ebb03cdac1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:08.804463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455e8db8-9236-4b34-a248-8267b598de88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:08.805040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e8db8-9236-4b34-a248-8267b598de88 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:08.805325 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e8db8-9236-4b34-a248-8267b598de88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:08.805599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455e8db8-9236-4b34-a248-8267b598de88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:08.815061 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e8db8-9236-4b34-a248-8267b598de88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:08.816546 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455e8db8-9236-4b34-a248-8267b598de88 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:08.825996 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 773/1536] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:06:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:06:11.039109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1bce5-4fa3-4a93-8d36-ff7724cc556f None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:11.040182 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1bce5-4fa3-4a93-8d36-ff7724cc556f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:11.040182 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d1bce5-4fa3-4a93-8d36-ff7724cc556f None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:11.042472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 764/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed May 25 16:06:11 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:06:11.047845 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df59f4d-4064-430b-85df-85e41a805d7e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:11.048300 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df59f4d-4064-430b-85df-85e41a805d7e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:11.048405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df59f4d-4064-430b-85df-85e41a805d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:11.067959 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9df59f4d-4064-430b-85df-85e41a805d7e None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:06:11.069195 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9df59f4d-4064-430b-85df-85e41a805d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:11.078345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 774/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed May 25 16:06:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:06:11.346921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1bce5-4fa3-4a93-8d36-ff7724cc556f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:06:11.348793 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-11d1bce5-4fa3-4a93-8d36-ff7724cc556f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:06:11.356952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=65 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:11.363901 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d1bce5-4fa3-4a93-8d36-ff7724cc556f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:06:11.368590 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f5d99cf-f4ed-4f1f-baf6-fa64fd99e6e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:06:11.369587 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d99cf-f4ed-4f1f-baf6-fa64fd99e6e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:06:11.369935 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d99cf-f4ed-4f1f-baf6-fa64fd99e6e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:06:11.370316 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f5d99cf-f4ed-4f1f-baf6-fa64fd99e6e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:06:11.385977 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d99cf-f4ed-4f1f-baf6-fa64fd99e6e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:06:11.387506 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f5d99cf-f4ed-4f1f-baf6-fa64fd99e6e9 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:06:11.397212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 765/1539] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:06:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:06:21.062522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=32 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:06:21.393126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=5 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:45.162130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d25dd6-687b-47d2-9719-2206c8771eee None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.162809 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d25dd6-687b-47d2-9719-2206c8771eee None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.163065 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0d25dd6-687b-47d2-9719-2206c8771eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.187278 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f0d25dd6-687b-47d2-9719-2206c8771eee None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:45.188534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f0d25dd6-687b-47d2-9719-2206c8771eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:45.202082 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 775/1540] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:45.213654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee23b1-4851-4a16-bff8-1440506c334f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.214597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee23b1-4851-4a16-bff8-1440506c334f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.215111 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ee23b1-4851-4a16-bff8-1440506c334f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.257720 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-38ee23b1-4851-4a16-bff8-1440506c334f None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:45.259309 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-38ee23b1-4851-4a16-bff8-1440506c334f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:45.271598 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 766/1541] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:45.277577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d25dd6-687b-47d2-9719-2206c8771eee None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:45.301908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0d25dd6-687b-47d2-9719-2206c8771eee None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:45.311861 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6460b-0d55-42bf-9dd5-ecd29c4c3bfe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:45.312573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6460b-0d55-42bf-9dd5-ecd29c4c3bfe admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.312868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6460b-0d55-42bf-9dd5-ecd29c4c3bfe admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.313157 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6460b-0d55-42bf-9dd5-ecd29c4c3bfe admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.313914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6460b-0d55-42bf-9dd5-ecd29c4c3bfe admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:45.314235 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6460b-0d55-42bf-9dd5-ecd29c4c3bfe admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:45.315965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6460b-0d55-42bf-9dd5-ecd29c4c3bfe admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:45.339654 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 776/1542] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:45.348817 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee23b1-4851-4a16-bff8-1440506c334f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:45.359674 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ee23b1-4851-4a16-bff8-1440506c334f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:45.370421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac70f76c-d42d-4505-9482-b4d6c009cffe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:45.371647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70f76c-d42d-4505-9482-b4d6c009cffe admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.372169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70f76c-d42d-4505-9482-b4d6c009cffe admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.372669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac70f76c-d42d-4505-9482-b4d6c009cffe admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.373590 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70f76c-d42d-4505-9482-b4d6c009cffe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:45.375929 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac70f76c-d42d-4505-9482-b4d6c009cffe admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:45.418130 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 767/1543] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:07:45 2022] POST /identity/v3/projects => generated 389 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:45.423273 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6460b-0d55-42bf-9dd5-ecd29c4c3bfe admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:45.440633 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6460b-0d55-42bf-9dd5-ecd29c4c3bfe admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:45.508435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c001a92-4703-4606-b91c-9909f1474169 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.509107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c001a92-4703-4606-b91c-9909f1474169 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.509648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c001a92-4703-4606-b91c-9909f1474169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.545699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-0c001a92-4703-4606-b91c-9909f1474169 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:45.551854 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e2fbe4-f7d1-48d1-a96f-a175034ba32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:45.551854 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2fbe4-f7d1-48d1-a96f-a175034ba32d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.551854 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2fbe4-f7d1-48d1-a96f-a175034ba32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.551854 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e2fbe4-f7d1-48d1-a96f-a175034ba32d admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.552384 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2fbe4-f7d1-48d1-a96f-a175034ba32d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:45.552929 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0c001a92-4703-4606-b91c-9909f1474169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:45.553783 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e2fbe4-f7d1-48d1-a96f-a175034ba32d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:45.566592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 768/1544] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 59 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:07:45.583172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bc6d4-203d-4345-9932-b3cabf278fb1 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.583872 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bc6d4-203d-4345-9932-b3cabf278fb1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.584443 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942bc6d4-203d-4345-9932-b3cabf278fb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.600483 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 777/1545] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:07:45 2022] POST /identity/v3/users => generated 295 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:45.601949 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd07268-ffca-4881-adb3-af1130f041d5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.602719 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd07268-ffca-4881-adb3-af1130f041d5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.603014 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd07268-ffca-4881-adb3-af1130f041d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.609213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-942bc6d4-203d-4345-9932-b3cabf278fb1 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:45.611394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-942bc6d4-203d-4345-9932-b3cabf278fb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:45.627065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 769/1546] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:45.628349 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bc6d4-203d-4345-9932-b3cabf278fb1 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:45.638539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-edd07268-ffca-4881-adb3-af1130f041d5 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:45.647858 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-edd07268-ffca-4881-adb3-af1130f041d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:45.662799 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942bc6d4-203d-4345-9932-b3cabf278fb1 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:45.673876 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f461dffe-5dfa-4418-b889-51da467e361e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:45.674981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461dffe-5dfa-4418-b889-51da467e361e admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.675493 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461dffe-5dfa-4418-b889-51da467e361e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.676007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f461dffe-5dfa-4418-b889-51da467e361e admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.676957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461dffe-5dfa-4418-b889-51da467e361e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:45.677271 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 778/1547] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:45.678560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd07268-ffca-4881-adb3-af1130f041d5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:45.679699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f461dffe-5dfa-4418-b889-51da467e361e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:45.692580 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd07268-ffca-4881-adb3-af1130f041d5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:45.702647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 770/1548] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:45.704949 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9021699-d4c2-4ab1-a687-532ec727efab None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.705501 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9021699-d4c2-4ab1-a687-532ec727efab None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.706030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9021699-d4c2-4ab1-a687-532ec727efab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.723221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ec1bf-bcd4-47a1-83b5-23dcb3b237da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:45.723221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ec1bf-bcd4-47a1-83b5-23dcb3b237da admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.723221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ec1bf-bcd4-47a1-83b5-23dcb3b237da admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.723221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ec1bf-bcd4-47a1-83b5-23dcb3b237da admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.723735 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ec1bf-bcd4-47a1-83b5-23dcb3b237da admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:45.723735 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ec1bf-bcd4-47a1-83b5-23dcb3b237da admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:45.723735 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ec1bf-bcd4-47a1-83b5-23dcb3b237da admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:45.731161 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f9021699-d4c2-4ab1-a687-532ec727efab None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:45.732738 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f9021699-d4c2-4ab1-a687-532ec727efab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:45.744599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 779/1549] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:45.748958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 771/1550] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:45.749339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ec1bf-bcd4-47a1-83b5-23dcb3b237da admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:45.753470 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9021699-d4c2-4ab1-a687-532ec727efab None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:45.767930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ec1bf-bcd4-47a1-83b5-23dcb3b237da admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:45.769591 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9021699-d4c2-4ab1-a687-532ec727efab None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:45.778950 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a029db-256b-4946-8d40-49115c95b5fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:45.779364 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b2e75e-ee4b-47cd-a11d-c30a9ee1636e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:45.780101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2e75e-ee4b-47cd-a11d-c30a9ee1636e admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.780460 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a029db-256b-4946-8d40-49115c95b5fa admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.780586 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2e75e-ee4b-47cd-a11d-c30a9ee1636e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.780804 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b2e75e-ee4b-47cd-a11d-c30a9ee1636e admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.781328 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a029db-256b-4946-8d40-49115c95b5fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.782171 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a029db-256b-4946-8d40-49115c95b5fa admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.782493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2e75e-ee4b-47cd-a11d-c30a9ee1636e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:45.783353 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a029db-256b-4946-8d40-49115c95b5fa admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:45.783893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a029db-256b-4946-8d40-49115c95b5fa admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:45.784163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b2e75e-ee4b-47cd-a11d-c30a9ee1636e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:45.785469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: WARNING py.warnings [None req-c6a029db-256b-4946-8d40-49115c95b5fa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required May 25 16:07:45.785469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: warnings.warn(msg) May 25 16:07:45.785469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: May 25 16:07:45.786848 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6a029db-256b-4946-8d40-49115c95b5fa admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:45.796734 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 780/1551] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:45 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:45.798278 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e999c8-a14a-471c-ae59-1558cd64aaef None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.798664 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e999c8-a14a-471c-ae59-1558cd64aaef None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.799048 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6e999c8-a14a-471c-ae59-1558cd64aaef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.800886 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 772/1552] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:07:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:45.802419 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e47805-840c-473f-bf12-b64397eb217c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.802950 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e47805-840c-473f-bf12-b64397eb217c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.803448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e47805-840c-473f-bf12-b64397eb217c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.824545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-95e47805-840c-473f-bf12-b64397eb217c None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:45.826521 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-95e47805-840c-473f-bf12-b64397eb217c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:45.827145 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c6e999c8-a14a-471c-ae59-1558cd64aaef None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:45.828563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c6e999c8-a14a-471c-ae59-1558cd64aaef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:45.835605 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 773/1553] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:45.836322 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 781/1554] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:45.837425 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e47805-840c-473f-bf12-b64397eb217c None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:45.837659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c92980-0280-47eb-bbc5-4805e03410de None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.844710 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c92980-0280-47eb-bbc5-4805e03410de None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.844969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c92980-0280-47eb-bbc5-4805e03410de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.850361 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e47805-840c-473f-bf12-b64397eb217c None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:45.855220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7b397-0ca5-4578-9a9a-0548a919fbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:45.855945 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7b397-0ca5-4578-9a9a-0548a919fbfa admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.856239 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7b397-0ca5-4578-9a9a-0548a919fbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.856527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83a7b397-0ca5-4578-9a9a-0548a919fbfa admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.857242 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7b397-0ca5-4578-9a9a-0548a919fbfa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:45.859210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83a7b397-0ca5-4578-9a9a-0548a919fbfa admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:45.865960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-22c92980-0280-47eb-bbc5-4805e03410de None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:45.867196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-22c92980-0280-47eb-bbc5-4805e03410de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:45.876054 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 782/1555] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:45.877249 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c92980-0280-47eb-bbc5-4805e03410de None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:45.889069 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c92980-0280-47eb-bbc5-4805e03410de None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:45.894047 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5253ce6a-c7f4-4a66-809f-4fe07cf03f9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:45.894991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253ce6a-c7f4-4a66-809f-4fe07cf03f9d admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.895279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253ce6a-c7f4-4a66-809f-4fe07cf03f9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.895555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5253ce6a-c7f4-4a66-809f-4fe07cf03f9d admin admin] PATH_INFO: `/v3/projects/feb4d81e1722439591065849449aeb18/users/be5c51fdd2fe4159a83a560dc074da6b/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.899744 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 774/1556] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:07:45 2022] POST /identity/v3/projects => generated 383 bytes in 60 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:45.901331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7b397-0ca5-4578-9a9a-0548a919fbfa admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:45.916699 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5253ce6a-c7f4-4a66-809f-4fe07cf03f9d admin admin] RBAC: Authorizing `identity:create_grant(project_id=feb4d81e1722439591065849449aeb18, user_id=be5c51fdd2fe4159a83a560dc074da6b, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:45.917791 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-5253ce6a-c7f4-4a66-809f-4fe07cf03f9d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:07:45.917791 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 25 16:07:45.917791 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: May 25 16:07:45.920791 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5253ce6a-c7f4-4a66-809f-4fe07cf03f9d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:45.939680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 783/1557] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:07:45 2022] PUT /identity/v3/projects/feb4d81e1722439591065849449aeb18/users/be5c51fdd2fe4159a83a560dc074da6b/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 63 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:07:45.945145 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7523c-16a2-4b17-8645-e544b0865e96 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:45.945514 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7523c-16a2-4b17-8645-e544b0865e96 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:45.945875 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7523c-16a2-4b17-8645-e544b0865e96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:45.965988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f3a7523c-16a2-4b17-8645-e544b0865e96 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:45.967311 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f3a7523c-16a2-4b17-8645-e544b0865e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:45.982757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83a7b397-0ca5-4578-9a9a-0548a919fbfa admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.027111 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 784/1558] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:46.028355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c3717-2677-4aeb-a82b-6f7dfbaef0d8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.028654 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c3717-2677-4aeb-a82b-6f7dfbaef0d8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.028936 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e8c3717-2677-4aeb-a82b-6f7dfbaef0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.036287 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:46.045592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef05013f-3194-49d4-89d4-14ff90393d82 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.046259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05013f-3194-49d4-89d4-14ff90393d82 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.046551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05013f-3194-49d4-89d4-14ff90393d82 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.046834 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef05013f-3194-49d4-89d4-14ff90393d82 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.047544 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05013f-3194-49d4-89d4-14ff90393d82 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.049494 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef05013f-3194-49d4-89d4-14ff90393d82 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.049792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7e8c3717-2677-4aeb-a82b-6f7dfbaef0d8 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:46.050985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7e8c3717-2677-4aeb-a82b-6f7dfbaef0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:46.058838 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 785/1559] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:46.059246 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:46.060072 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae6bff-daef-4cb8-b2f3-a9a879991085 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.060360 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae6bff-daef-4cb8-b2f3-a9a879991085 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.060639 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caae6bff-daef-4cb8-b2f3-a9a879991085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.079938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-caae6bff-daef-4cb8-b2f3-a9a879991085 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:46.081072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 775/1560] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:07:45 2022] POST /identity/v3/projects => generated 391 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:46.081201 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-caae6bff-daef-4cb8-b2f3-a9a879991085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:46.082446 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc431d8-7f85-40d8-a0b3-ee08e01f22e1 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.082750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc431d8-7f85-40d8-a0b3-ee08e01f22e1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.083230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc431d8-7f85-40d8-a0b3-ee08e01f22e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.100889 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4cc431d8-7f85-40d8-a0b3-ee08e01f22e1 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:46.102087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4cc431d8-7f85-40d8-a0b3-ee08e01f22e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:46.144298 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 786/1561] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:46.145129 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae6bff-daef-4cb8-b2f3-a9a879991085 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.154470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caae6bff-daef-4cb8-b2f3-a9a879991085 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.159440 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-485a69c7-1c7c-489e-9564-a3c6f5b7ec75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.160130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a69c7-1c7c-489e-9564-a3c6f5b7ec75 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.160427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a69c7-1c7c-489e-9564-a3c6f5b7ec75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.160710 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-485a69c7-1c7c-489e-9564-a3c6f5b7ec75 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.161403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a69c7-1c7c-489e-9564-a3c6f5b7ec75 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.162948 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 776/1562] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:46.163363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-485a69c7-1c7c-489e-9564-a3c6f5b7ec75 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.168150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365a48b-6596-4400-aecb-32b15cd52189 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.168471 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365a48b-6596-4400-aecb-32b15cd52189 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.168761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2365a48b-6596-4400-aecb-32b15cd52189 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.187051 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 787/1563] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/users => generated 292 bytes in 39 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:46.187051 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d6645-b45e-4893-9cb4-e0a79267f662 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.187051 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d6645-b45e-4893-9cb4-e0a79267f662 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.187051 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512d6645-b45e-4893-9cb4-e0a79267f662 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.187568 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-2365a48b-6596-4400-aecb-32b15cd52189 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:46.187655 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2365a48b-6596-4400-aecb-32b15cd52189 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:46.195792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 777/1564] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:46.195792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d41635-5546-4152-a561-ca88c20a427e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.195792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d41635-5546-4152-a561-ca88c20a427e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.195792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34d41635-5546-4152-a561-ca88c20a427e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.211454 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-512d6645-b45e-4893-9cb4-e0a79267f662 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:46.212940 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-512d6645-b45e-4893-9cb4-e0a79267f662 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:46.217469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-34d41635-5546-4152-a561-ca88c20a427e None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:46.218686 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-34d41635-5546-4152-a561-ca88c20a427e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:46.221448 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 788/1565] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:46.224093 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d6645-b45e-4893-9cb4-e0a79267f662 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.234240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512d6645-b45e-4893-9cb4-e0a79267f662 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.234636 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 778/1566] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:46.235522 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d41635-5546-4152-a561-ca88c20a427e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.242807 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d29f3-1d7f-4ff0-88f4-7f5dfd4ebb6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.243716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d29f3-1d7f-4ff0-88f4-7f5dfd4ebb6d admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.244282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d29f3-1d7f-4ff0-88f4-7f5dfd4ebb6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.244722 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d29f3-1d7f-4ff0-88f4-7f5dfd4ebb6d admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.245589 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34d41635-5546-4152-a561-ca88c20a427e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.245904 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d29f3-1d7f-4ff0-88f4-7f5dfd4ebb6d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:46.246376 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d29f3-1d7f-4ff0-88f4-7f5dfd4ebb6d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.248643 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-658d29f3-1d7f-4ff0-88f4-7f5dfd4ebb6d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.250246 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a2008-074c-40d2-b54f-029aef48f079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.250920 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a2008-074c-40d2-b54f-029aef48f079 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.251211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a2008-074c-40d2-b54f-029aef48f079 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.251492 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-125a2008-074c-40d2-b54f-029aef48f079 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.252230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a2008-074c-40d2-b54f-029aef48f079 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.253267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: WARNING py.warnings [None req-125a2008-074c-40d2-b54f-029aef48f079 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:07:46.253267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: warnings.warn(msg) May 25 16:07:46.253267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: May 25 16:07:46.254472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-125a2008-074c-40d2-b54f-029aef48f079 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.263933 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 789/1567] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:46.265114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d29f3-1d7f-4ff0-88f4-7f5dfd4ebb6d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.275364 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-658d29f3-1d7f-4ff0-88f4-7f5dfd4ebb6d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.280263 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53545ef-c38c-4501-90b9-bd10b569b161 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.280891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53545ef-c38c-4501-90b9-bd10b569b161 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.283570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53545ef-c38c-4501-90b9-bd10b569b161 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.283909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53545ef-c38c-4501-90b9-bd10b569b161 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.288015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 779/1568] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/users => generated 296 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:46.288156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53545ef-c38c-4501-90b9-bd10b569b161 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:46.288156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53545ef-c38c-4501-90b9-bd10b569b161 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.288338 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a2008-074c-40d2-b54f-029aef48f079 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.288423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e53545ef-c38c-4501-90b9-bd10b569b161 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.302031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-125a2008-074c-40d2-b54f-029aef48f079 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.306949 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee947981-67e0-48c3-b8be-71cfa6891356 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.307630 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee947981-67e0-48c3-b8be-71cfa6891356 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.307952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee947981-67e0-48c3-b8be-71cfa6891356 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.308181 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 790/1569] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:46.308297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee947981-67e0-48c3-b8be-71cfa6891356 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.309223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee947981-67e0-48c3-b8be-71cfa6891356 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:46.309506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53545ef-c38c-4501-90b9-bd10b569b161 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.309624 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee947981-67e0-48c3-b8be-71cfa6891356 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.311124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee947981-67e0-48c3-b8be-71cfa6891356 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.320693 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53545ef-c38c-4501-90b9-bd10b569b161 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.323773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 780/1570] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:46.324703 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee947981-67e0-48c3-b8be-71cfa6891356 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.333404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0edf9bbf-3391-490b-b85c-574ddc22deaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.334103 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf9bbf-3391-490b-b85c-574ddc22deaa admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.334396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf9bbf-3391-490b-b85c-574ddc22deaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.334678 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0edf9bbf-3391-490b-b85c-574ddc22deaa admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.335419 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf9bbf-3391-490b-b85c-574ddc22deaa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.337102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0edf9bbf-3391-490b-b85c-574ddc22deaa admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.348118 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 791/1571] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:46.349425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd0b9f-bf8c-49ef-9263-3c47de63c915 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.349721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd0b9f-bf8c-49ef-9263-3c47de63c915 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.350005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cd0b9f-bf8c-49ef-9263-3c47de63c915 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.368943 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-26cd0b9f-bf8c-49ef-9263-3c47de63c915 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:46.370440 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-26cd0b9f-bf8c-49ef-9263-3c47de63c915 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:46.378529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 792/1572] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:46.382678 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd0b9f-bf8c-49ef-9263-3c47de63c915 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.386988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee947981-67e0-48c3-b8be-71cfa6891356 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.389357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cd0b9f-bf8c-49ef-9263-3c47de63c915 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.390541 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef38d0-78d7-4aad-b49d-b8175d3b9658 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.391202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef38d0-78d7-4aad-b49d-b8175d3b9658 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.391486 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef38d0-78d7-4aad-b49d-b8175d3b9658 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.391761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ef38d0-78d7-4aad-b49d-b8175d3b9658 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.393880 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2185c-3075-4e1c-bc32-5b1aed20b838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.394559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2185c-3075-4e1c-bc32-5b1aed20b838 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.394932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2185c-3075-4e1c-bc32-5b1aed20b838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.395233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc2185c-3075-4e1c-bc32-5b1aed20b838 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.396009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2185c-3075-4e1c-bc32-5b1aed20b838 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:46.396351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2185c-3075-4e1c-bc32-5b1aed20b838 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.398001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc2185c-3075-4e1c-bc32-5b1aed20b838 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.400938 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef38d0-78d7-4aad-b49d-b8175d3b9658 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.402359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ef38d0-78d7-4aad-b49d-b8175d3b9658 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.417401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 793/1573] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:46.417401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2185c-3075-4e1c-bc32-5b1aed20b838 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.417713 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 781/1574] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:07:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:07:46.417713 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef38d0-78d7-4aad-b49d-b8175d3b9658 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.421644 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc2185c-3075-4e1c-bc32-5b1aed20b838 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.422835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ef38d0-78d7-4aad-b49d-b8175d3b9658 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.426650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe136a-a405-41fa-b875-a5d735bdb49d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.427319 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe136a-a405-41fa-b875-a5d735bdb49d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.427631 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe136a-a405-41fa-b875-a5d735bdb49d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.427932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfe136a-a405-41fa-b875-a5d735bdb49d admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.428667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe136a-a405-41fa-b875-a5d735bdb49d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.429065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02afed53-8bf0-4826-9efe-dbd90f6d57ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.429706 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-0bfe136a-a405-41fa-b875-a5d735bdb49d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:07:46.429706 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 25 16:07:46.429706 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: May 25 16:07:46.430429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afed53-8bf0-4826-9efe-dbd90f6d57ba admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.430907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfe136a-a405-41fa-b875-a5d735bdb49d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.431089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afed53-8bf0-4826-9efe-dbd90f6d57ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.431523 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02afed53-8bf0-4826-9efe-dbd90f6d57ba admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.432521 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02afed53-8bf0-4826-9efe-dbd90f6d57ba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.434320 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02afed53-8bf0-4826-9efe-dbd90f6d57ba admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.446168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 782/1575] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:46.450623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02afed53-8bf0-4826-9efe-dbd90f6d57ba admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.460547 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 794/1576] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/projects => generated 385 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:46.461553 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe136a-a405-41fa-b875-a5d735bdb49d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.464257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02afed53-8bf0-4826-9efe-dbd90f6d57ba admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.508532 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:46.521186 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:46.531213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff74b9-b508-429f-8cda-632438a9b9a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.532204 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff74b9-b508-429f-8cda-632438a9b9a6 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.532500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff74b9-b508-429f-8cda-632438a9b9a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.533219 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ff74b9-b508-429f-8cda-632438a9b9a6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.533530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff74b9-b508-429f-8cda-632438a9b9a6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.535500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ff74b9-b508-429f-8cda-632438a9b9a6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.536729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfe136a-a405-41fa-b875-a5d735bdb49d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.543288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d8d26-286b-4707-91b6-d2e5b4028202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.544065 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d8d26-286b-4707-91b6-d2e5b4028202 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.544639 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d8d26-286b-4707-91b6-d2e5b4028202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.544785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d8d26-286b-4707-91b6-d2e5b4028202 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.546208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d8d26-286b-4707-91b6-d2e5b4028202 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.549112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874d8d26-286b-4707-91b6-d2e5b4028202 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.567250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 783/1577] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/projects => generated 373 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:46.568281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff74b9-b508-429f-8cda-632438a9b9a6 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.581434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 795/1578] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/projects => generated 375 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:46.582567 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d8d26-286b-4707-91b6-d2e5b4028202 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.660114 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ff74b9-b508-429f-8cda-632438a9b9a6 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.665430 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad93e5-8431-4ddc-9078-fb994912dfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.666178 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad93e5-8431-4ddc-9078-fb994912dfec admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.666475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad93e5-8431-4ddc-9078-fb994912dfec admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.666761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20ad93e5-8431-4ddc-9078-fb994912dfec admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.667701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad93e5-8431-4ddc-9078-fb994912dfec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.669390 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20ad93e5-8431-4ddc-9078-fb994912dfec admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.670256 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d8d26-286b-4707-91b6-d2e5b4028202 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.675266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244fc2c-e8c1-46f1-b184-5ecac410f232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.675812 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244fc2c-e8c1-46f1-b184-5ecac410f232 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.676260 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244fc2c-e8c1-46f1-b184-5ecac410f232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.676551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b244fc2c-e8c1-46f1-b184-5ecac410f232 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.677281 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244fc2c-e8c1-46f1-b184-5ecac410f232 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:07:46.677595 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244fc2c-e8c1-46f1-b184-5ecac410f232 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.679458 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b244fc2c-e8c1-46f1-b184-5ecac410f232 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.680275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 784/1579] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:46.681074 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad93e5-8431-4ddc-9078-fb994912dfec admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.691521 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20ad93e5-8431-4ddc-9078-fb994912dfec admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.701376 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 796/1580] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:07:46.701376 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244fc2c-e8c1-46f1-b184-5ecac410f232 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.705287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6d4d2-1429-470c-ade5-0170fb2a4f0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.706009 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6d4d2-1429-470c-ade5-0170fb2a4f0a admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.706295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6d4d2-1429-470c-ade5-0170fb2a4f0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.706570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc6d4d2-1429-470c-ade5-0170fb2a4f0a admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.707494 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6d4d2-1429-470c-ade5-0170fb2a4f0a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.710166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc6d4d2-1429-470c-ade5-0170fb2a4f0a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.712449 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b244fc2c-e8c1-46f1-b184-5ecac410f232 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.717325 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b4799-8224-4335-9fc3-fdee4a1cfda3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.718006 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b4799-8224-4335-9fc3-fdee4a1cfda3 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.718302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b4799-8224-4335-9fc3-fdee4a1cfda3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.718969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7b4799-8224-4335-9fc3-fdee4a1cfda3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.719460 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b4799-8224-4335-9fc3-fdee4a1cfda3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.721119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7b4799-8224-4335-9fc3-fdee4a1cfda3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.735910 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 797/1581] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:46.735910 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b4799-8224-4335-9fc3-fdee4a1cfda3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.741487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 785/1582] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:46.741487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6d4d2-1429-470c-ade5-0170fb2a4f0a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.748021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7b4799-8224-4335-9fc3-fdee4a1cfda3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.812767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40913a-5488-4441-8b0d-e2417cf0ebe6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.813580 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40913a-5488-4441-8b0d-e2417cf0ebe6 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.813932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40913a-5488-4441-8b0d-e2417cf0ebe6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.814284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a40913a-5488-4441-8b0d-e2417cf0ebe6 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.815421 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40913a-5488-4441-8b0d-e2417cf0ebe6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.815537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc6d4d2-1429-470c-ade5-0170fb2a4f0a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.817135 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a40913a-5488-4441-8b0d-e2417cf0ebe6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.820392 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186ac7c-5e0d-4fe9-86b9-bd993bd382ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.821166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186ac7c-5e0d-4fe9-86b9-bd993bd382ab admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.821889 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186ac7c-5e0d-4fe9-86b9-bd993bd382ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.821889 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c186ac7c-5e0d-4fe9-86b9-bd993bd382ab admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.822758 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186ac7c-5e0d-4fe9-86b9-bd993bd382ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.824878 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c186ac7c-5e0d-4fe9-86b9-bd993bd382ab admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.842666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 798/1583] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/users => generated 293 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:46.842666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40913a-5488-4441-8b0d-e2417cf0ebe6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.854802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 786/1584] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/users => generated 287 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:46.854802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186ac7c-5e0d-4fe9-86b9-bd993bd382ab admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.855145 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a40913a-5488-4441-8b0d-e2417cf0ebe6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.860941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08fa7a-4400-40e8-88c6-3d98e39c5645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.861833 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08fa7a-4400-40e8-88c6-3d98e39c5645 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.862664 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08fa7a-4400-40e8-88c6-3d98e39c5645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.862953 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a08fa7a-4400-40e8-88c6-3d98e39c5645 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.863986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08fa7a-4400-40e8-88c6-3d98e39c5645 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.866097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a08fa7a-4400-40e8-88c6-3d98e39c5645 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.866625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c186ac7c-5e0d-4fe9-86b9-bd993bd382ab admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.872183 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1ac2a-9496-4a4f-8fe3-f536660b1456 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:46.873319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1ac2a-9496-4a4f-8fe3-f536660b1456 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:46.873807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1ac2a-9496-4a4f-8fe3-f536660b1456 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:46.874189 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b1ac2a-9496-4a4f-8fe3-f536660b1456 admin admin] PATH_INFO: `/v3/projects/32e48e8e884b4840ba54d546d82d2334/users/708fdc537e1747d7bd5917ed7af9c368/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:46.893871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1ac2a-9496-4a4f-8fe3-f536660b1456 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32e48e8e884b4840ba54d546d82d2334, user_id=708fdc537e1747d7bd5917ed7af9c368, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:46.896826 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b1ac2a-9496-4a4f-8fe3-f536660b1456 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:46.906066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 799/1585] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/users => generated 288 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:46.907308 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08fa7a-4400-40e8-88c6-3d98e39c5645 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.917475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a08fa7a-4400-40e8-88c6-3d98e39c5645 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:46.917789 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 787/1586] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:07:46 2022] PUT /identity/v3/projects/32e48e8e884b4840ba54d546d82d2334/users/708fdc537e1747d7bd5917ed7af9c368/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:07:46.921619 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1ac2a-9496-4a4f-8fe3-f536660b1456 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:46.927155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:46.963483 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:46.999965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6e843-6572-48b4-a384-c77ea4d0dbd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.000650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6e843-6572-48b4-a384-c77ea4d0dbd3 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.000938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6e843-6572-48b4-a384-c77ea4d0dbd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.001227 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6e843-6572-48b4-a384-c77ea4d0dbd3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.001936 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f6e843-6572-48b4-a384-c77ea4d0dbd3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.004093 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f6e843-6572-48b4-a384-c77ea4d0dbd3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.005850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b1ac2a-9496-4a4f-8fe3-f536660b1456 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.011266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022855a-252d-4cbc-9a91-081454a97dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.012602 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022855a-252d-4cbc-9a91-081454a97dc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.012899 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022855a-252d-4cbc-9a91-081454a97dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.013185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4022855a-252d-4cbc-9a91-081454a97dc0 admin admin] PATH_INFO: `/v3/projects/f7733d8b5059433eb18dc6b174eb9533/users/d0427d9cce40423185a40df61c3617a6/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.030231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022855a-252d-4cbc-9a91-081454a97dc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7733d8b5059433eb18dc6b174eb9533, user_id=d0427d9cce40423185a40df61c3617a6, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.033254 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 800/1587] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:07:46 2022] POST /identity/v3/projects => generated 381 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:47.033379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4022855a-252d-4cbc-9a91-081454a97dc0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.034539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6e843-6572-48b4-a384-c77ea4d0dbd3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.056235 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 788/1588] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:07:46 2022] PUT /identity/v3/projects/f7733d8b5059433eb18dc6b174eb9533/users/d0427d9cce40423185a40df61c3617a6/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:07:47.057989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022855a-252d-4cbc-9a91-081454a97dc0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.106712 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f6e843-6572-48b4-a384-c77ea4d0dbd3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.129946 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4022855a-252d-4cbc-9a91-081454a97dc0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.135725 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb72d35-c2bf-4a63-988e-f8698d3d346c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.136460 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb72d35-c2bf-4a63-988e-f8698d3d346c admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.136757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb72d35-c2bf-4a63-988e-f8698d3d346c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.137049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb72d35-c2bf-4a63-988e-f8698d3d346c admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.137789 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb72d35-c2bf-4a63-988e-f8698d3d346c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.139460 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb72d35-c2bf-4a63-988e-f8698d3d346c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.151733 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 789/1589] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:47.153305 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb72d35-c2bf-4a63-988e-f8698d3d346c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.165002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb72d35-c2bf-4a63-988e-f8698d3d346c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.171994 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82aba1-93cf-44c9-b3d7-8f497a37fd8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.172806 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82aba1-93cf-44c9-b3d7-8f497a37fd8d admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.173156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82aba1-93cf-44c9-b3d7-8f497a37fd8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.173501 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b82aba1-93cf-44c9-b3d7-8f497a37fd8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.174348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82aba1-93cf-44c9-b3d7-8f497a37fd8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.176273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b82aba1-93cf-44c9-b3d7-8f497a37fd8d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.190754 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 790/1590] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:47.191835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82aba1-93cf-44c9-b3d7-8f497a37fd8d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.192168 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fef318e-44ca-4e7b-b832-5c462dc186fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.194119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef318e-44ca-4e7b-b832-5c462dc186fc admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.194470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef318e-44ca-4e7b-b832-5c462dc186fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.194819 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fef318e-44ca-4e7b-b832-5c462dc186fc admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.195532 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef318e-44ca-4e7b-b832-5c462dc186fc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.198588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fef318e-44ca-4e7b-b832-5c462dc186fc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.207874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b82aba1-93cf-44c9-b3d7-8f497a37fd8d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.214589 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115b325-cc6f-4728-a0f1-ce099e69a921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.215873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115b325-cc6f-4728-a0f1-ce099e69a921 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.216198 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115b325-cc6f-4728-a0f1-ce099e69a921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.216473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6115b325-cc6f-4728-a0f1-ce099e69a921 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.217623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115b325-cc6f-4728-a0f1-ce099e69a921 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.219305 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6115b325-cc6f-4728-a0f1-ce099e69a921 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.233359 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 801/1591] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:07:47 2022] POST /identity/v3/users => generated 290 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:47.233359 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fb72d-e5c9-40e2-b26c-cfc29adfc419 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.233359 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fb72d-e5c9-40e2-b26c-cfc29adfc419 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.233715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 791/1592] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:47.233825 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5fb72d-e5c9-40e2-b26c-cfc29adfc419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.233925 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115b325-cc6f-4728-a0f1-ce099e69a921 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.245001 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6115b325-cc6f-4728-a0f1-ce099e69a921 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.251177 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166981ba-e552-45cf-bdb6-34fcaed60059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.252034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166981ba-e552-45cf-bdb6-34fcaed60059 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.252583 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166981ba-e552-45cf-bdb6-34fcaed60059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.252840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-166981ba-e552-45cf-bdb6-34fcaed60059 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.253567 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166981ba-e552-45cf-bdb6-34fcaed60059 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.255651 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-166981ba-e552-45cf-bdb6-34fcaed60059 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.258816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5c5fb72d-e5c9-40e2-b26c-cfc29adfc419 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:47.260053 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5c5fb72d-e5c9-40e2-b26c-cfc29adfc419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:47.272156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 802/1593] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:47.280424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9ae31-6de2-4fa6-8900-59817a90fd72 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.282675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9ae31-6de2-4fa6-8900-59817a90fd72 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.283013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe9ae31-6de2-4fa6-8900-59817a90fd72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.294118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 792/1594] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:07:47 2022] POST /identity/v3/users => generated 291 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:47.294118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166981ba-e552-45cf-bdb6-34fcaed60059 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.304753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-166981ba-e552-45cf-bdb6-34fcaed60059 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.311098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cff3ff-d8bb-44b7-8207-a149f543b6d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.311855 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cff3ff-d8bb-44b7-8207-a149f543b6d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.312153 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cff3ff-d8bb-44b7-8207-a149f543b6d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.312433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cff3ff-d8bb-44b7-8207-a149f543b6d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.313223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cff3ff-d8bb-44b7-8207-a149f543b6d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.314862 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cff3ff-d8bb-44b7-8207-a149f543b6d2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.316286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bfe9ae31-6de2-4fa6-8900-59817a90fd72 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:47.317782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bfe9ae31-6de2-4fa6-8900-59817a90fd72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:47.326503 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 803/1595] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:47.327771 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9ae31-6de2-4fa6-8900-59817a90fd72 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.328590 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 793/1596] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:47.329611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cff3ff-d8bb-44b7-8207-a149f543b6d2 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.343377 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cff3ff-d8bb-44b7-8207-a149f543b6d2 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.348678 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688ddcb-6167-4382-9cf5-fe8f5bdefd29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.349472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688ddcb-6167-4382-9cf5-fe8f5bdefd29 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.349958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688ddcb-6167-4382-9cf5-fe8f5bdefd29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.350255 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f688ddcb-6167-4382-9cf5-fe8f5bdefd29 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.350991 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688ddcb-6167-4382-9cf5-fe8f5bdefd29 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.353912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f688ddcb-6167-4382-9cf5-fe8f5bdefd29 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.355791 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe9ae31-6de2-4fa6-8900-59817a90fd72 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.361103 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e5f9c-f677-4499-a28a-0c86159148da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.361984 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e5f9c-f677-4499-a28a-0c86159148da admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.362376 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e5f9c-f677-4499-a28a-0c86159148da admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.362777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198e5f9c-f677-4499-a28a-0c86159148da admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.363622 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e5f9c-f677-4499-a28a-0c86159148da admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.368271 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198e5f9c-f677-4499-a28a-0c86159148da admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.368417 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 794/1597] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:47.368417 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688ddcb-6167-4382-9cf5-fe8f5bdefd29 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.380954 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 804/1598] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:47.381141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f688ddcb-6167-4382-9cf5-fe8f5bdefd29 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.381295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e5f9c-f677-4499-a28a-0c86159148da admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.381635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-198e5f9c-f677-4499-a28a-0c86159148da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:47.384292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff014be8-786c-4713-90f3-2e86888cbdcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.385143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff014be8-786c-4713-90f3-2e86888cbdcf admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.385644 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff014be8-786c-4713-90f3-2e86888cbdcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.385921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff014be8-786c-4713-90f3-2e86888cbdcf admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.386696 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff014be8-786c-4713-90f3-2e86888cbdcf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.389160 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff014be8-786c-4713-90f3-2e86888cbdcf admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.401870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 795/1599] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:47.403311 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff014be8-786c-4713-90f3-2e86888cbdcf admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.416956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff014be8-786c-4713-90f3-2e86888cbdcf admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.422656 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31594f09-3409-41be-a9f0-1d8e372c4d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.423689 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31594f09-3409-41be-a9f0-1d8e372c4d06 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.424010 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31594f09-3409-41be-a9f0-1d8e372c4d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.424672 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:47.425314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31594f09-3409-41be-a9f0-1d8e372c4d06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.425782 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31594f09-3409-41be-a9f0-1d8e372c4d06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.428488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31594f09-3409-41be-a9f0-1d8e372c4d06 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.435161 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:47.447291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 796/1600] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:47.448473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31594f09-3409-41be-a9f0-1d8e372c4d06 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.458792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198e5f9c-f677-4499-a28a-0c86159148da admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.459291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31594f09-3409-41be-a9f0-1d8e372c4d06 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.464265 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8720561-8af6-40d7-93bf-e01bf7591b53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.464479 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a772d5-e188-4950-81be-d1d8fd8fbf90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.465081 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8720561-8af6-40d7-93bf-e01bf7591b53 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.465384 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a772d5-e188-4950-81be-d1d8fd8fbf90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.465508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8720561-8af6-40d7-93bf-e01bf7591b53 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.465670 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a772d5-e188-4950-81be-d1d8fd8fbf90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.465897 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8720561-8af6-40d7-93bf-e01bf7591b53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.466016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a772d5-e188-4950-81be-d1d8fd8fbf90 admin admin] PATH_INFO: `/v3/projects/04d735ad2d88494f95cda019507da1df/users/e970b42de0dd4ecbb11bfa8401d234e5/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.479119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8720561-8af6-40d7-93bf-e01bf7591b53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.480951 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8720561-8af6-40d7-93bf-e01bf7591b53 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.482413 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a772d5-e188-4950-81be-d1d8fd8fbf90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04d735ad2d88494f95cda019507da1df, user_id=e970b42de0dd4ecbb11bfa8401d234e5, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.485489 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26a772d5-e188-4950-81be-d1d8fd8fbf90 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.494837 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 805/1601] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:07:47.503846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 797/1602] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:07:47 2022] PUT /identity/v3/projects/04d735ad2d88494f95cda019507da1df/users/e970b42de0dd4ecbb11bfa8401d234e5/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:07:47.509466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a772d5-e188-4950-81be-d1d8fd8fbf90 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.510790 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8720561-8af6-40d7-93bf-e01bf7591b53 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.523060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8720561-8af6-40d7-93bf-e01bf7591b53 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.590222 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a772d5-e188-4950-81be-d1d8fd8fbf90 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.595285 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43ef56-b8a1-40ad-8a5a-0580785f43c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.596133 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddbe8a-119a-421e-9979-bb91c8863bee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.596274 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43ef56-b8a1-40ad-8a5a-0580785f43c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.596580 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43ef56-b8a1-40ad-8a5a-0580785f43c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.597015 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d43ef56-b8a1-40ad-8a5a-0580785f43c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.598006 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddbe8a-119a-421e-9979-bb91c8863bee admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.598490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddbe8a-119a-421e-9979-bb91c8863bee admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.598955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ddbe8a-119a-421e-9979-bb91c8863bee admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.600066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddbe8a-119a-421e-9979-bb91c8863bee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.601962 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ddbe8a-119a-421e-9979-bb91c8863bee admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.609781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43ef56-b8a1-40ad-8a5a-0580785f43c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.612000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d43ef56-b8a1-40ad-8a5a-0580785f43c8 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.614978 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 798/1603] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:47.616487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddbe8a-119a-421e-9979-bb91c8863bee admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.624713 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 806/1604] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:07:47.626028 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43ef56-b8a1-40ad-8a5a-0580785f43c8 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.629603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ddbe8a-119a-421e-9979-bb91c8863bee admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.635405 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230e19a-8184-49fb-96e6-b06ddfc70962 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.636818 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230e19a-8184-49fb-96e6-b06ddfc70962 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.637874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230e19a-8184-49fb-96e6-b06ddfc70962 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.638371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a230e19a-8184-49fb-96e6-b06ddfc70962 admin admin] PATH_INFO: `/v3/projects/0b10cd68403a4ef7888e6ed340a04916/users/09ea4058ec5f4cbea82069ee235ef911/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.639522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d43ef56-b8a1-40ad-8a5a-0580785f43c8 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.657101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abdf67c2-e253-4176-8585-6fcdc37723a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.657651 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230e19a-8184-49fb-96e6-b06ddfc70962 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0b10cd68403a4ef7888e6ed340a04916, user_id=09ea4058ec5f4cbea82069ee235ef911, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.658066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf67c2-e253-4176-8585-6fcdc37723a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.658485 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf67c2-e253-4176-8585-6fcdc37723a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.659321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abdf67c2-e253-4176-8585-6fcdc37723a6 admin admin] PATH_INFO: `/v3/projects/93946fd1575e45428d2388015d64628f/users/a0289402c6f245839a86b7b66714a8f6/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.661825 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a230e19a-8184-49fb-96e6-b06ddfc70962 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.688962 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 799/1605] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:07:47 2022] PUT /identity/v3/projects/0b10cd68403a4ef7888e6ed340a04916/users/09ea4058ec5f4cbea82069ee235ef911/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:07:47.690249 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230e19a-8184-49fb-96e6-b06ddfc70962 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.707110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf67c2-e253-4176-8585-6fcdc37723a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93946fd1575e45428d2388015d64628f, user_id=a0289402c6f245839a86b7b66714a8f6, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.710681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abdf67c2-e253-4176-8585-6fcdc37723a6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.737856 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 807/1606] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:07:47 2022] PUT /identity/v3/projects/93946fd1575e45428d2388015d64628f/users/a0289402c6f245839a86b7b66714a8f6/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:07:47.739851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf97a2-e62d-493f-a928-517c1f2b7f6c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.740367 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf97a2-e62d-493f-a928-517c1f2b7f6c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.740895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cf97a2-e62d-493f-a928-517c1f2b7f6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.782197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-14cf97a2-e62d-493f-a928-517c1f2b7f6c None None] MFA Rules not processed for user `e970b42de0dd4ecbb11bfa8401d234e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:47.784089 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-14cf97a2-e62d-493f-a928-517c1f2b7f6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:47.794178 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a230e19a-8184-49fb-96e6-b06ddfc70962 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.862426 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:47.874140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9798c-7fb2-4825-8fee-e62907b4bb16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.874620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9798c-7fb2-4825-8fee-e62907b4bb16 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.874896 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9798c-7fb2-4825-8fee-e62907b4bb16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.875324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba9798c-7fb2-4825-8fee-e62907b4bb16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.876100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9798c-7fb2-4825-8fee-e62907b4bb16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.877921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ba9798c-7fb2-4825-8fee-e62907b4bb16 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.897083 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 800/1607] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:47 2022] GET /identity/v3/roles => generated 1420 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:47.898161 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9798c-7fb2-4825-8fee-e62907b4bb16 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:47.913721 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba9798c-7fb2-4825-8fee-e62907b4bb16 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:47.920693 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b8a4c8-1bbb-4326-a1bd-f82aa4d33d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:47.922301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8a4c8-1bbb-4326-a1bd-f82aa4d33d4d admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.922692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8a4c8-1bbb-4326-a1bd-f82aa4d33d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.923048 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b8a4c8-1bbb-4326-a1bd-f82aa4d33d4d admin admin] PATH_INFO: `/v3/projects/1f6d103cb389478f94ce6afe687c0e28/users/1d4d043b13da4b17830481f4a8da9b79/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.935066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 808/1608] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:47.937336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8161442-076b-4eda-8271-95b347380013 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:47.937809 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8161442-076b-4eda-8271-95b347380013 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:47.938226 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8161442-076b-4eda-8271-95b347380013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:47.976166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8a4c8-1bbb-4326-a1bd-f82aa4d33d4d admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f6d103cb389478f94ce6afe687c0e28, user_id=1d4d043b13da4b17830481f4a8da9b79, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:47.979378 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34b8a4c8-1bbb-4326-a1bd-f82aa4d33d4d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:47.986133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f8161442-076b-4eda-8271-95b347380013 None None] MFA Rules not processed for user `09ea4058ec5f4cbea82069ee235ef911`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:47.987644 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f8161442-076b-4eda-8271-95b347380013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:48.006725 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 801/1609] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:07:47 2022] PUT /identity/v3/projects/1f6d103cb389478f94ce6afe687c0e28/users/1d4d043b13da4b17830481f4a8da9b79/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:07:48.009843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fefa6-55e3-4904-b439-677fd75968ca None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.011690 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fefa6-55e3-4904-b439-677fd75968ca None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.012172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-147fefa6-55e3-4904-b439-677fd75968ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.056596 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:48.057729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-147fefa6-55e3-4904-b439-677fd75968ca None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:48.061853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-147fefa6-55e3-4904-b439-677fd75968ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:48.113572 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 809/1610] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:48.114645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8161442-076b-4eda-8271-95b347380013 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:48.136693 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8161442-076b-4eda-8271-95b347380013 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:48.173088 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 802/1611] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:48.180561 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c14f30-6ba8-48f7-90f6-cd92614d87ab None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.182138 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c14f30-6ba8-48f7-90f6-cd92614d87ab None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.183341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14c14f30-6ba8-48f7-90f6-cd92614d87ab None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.188460 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 803/1612] 10.222.0.85 () {54 vars in 961 bytes} [Wed May 25 16:07:48 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:48.190401 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87586d-d4aa-4201-9d8d-bc58c0ea33ce None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.192819 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87586d-d4aa-4201-9d8d-bc58c0ea33ce None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.193224 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de87586d-d4aa-4201-9d8d-bc58c0ea33ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.227318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-de87586d-d4aa-4201-9d8d-bc58c0ea33ce None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:48.227318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-de87586d-d4aa-4201-9d8d-bc58c0ea33ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:48.240309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186ead8-4f66-4d03-ad91-5c49785e3ec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:48.241340 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186ead8-4f66-4d03-ad91-5c49785e3ec7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.241696 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186ead8-4f66-4d03-ad91-5c49785e3ec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.242048 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a186ead8-4f66-4d03-ad91-5c49785e3ec7 admin admin] PATH_INFO: `/v3/projects/072316d170de4f50a27886d804fa252a/users/84a88a406e7349478f31ee93b258eca0/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.248116 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 804/1613] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:48.248116 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fc26b-6d2a-468f-bc6e-d30c97bca6e9 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.248116 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fc26b-6d2a-468f-bc6e-d30c97bca6e9 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.248116 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54fc26b-6d2a-468f-bc6e-d30c97bca6e9 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.252213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 805/1614] 10.222.0.85 () {54 vars in 961 bytes} [Wed May 25 16:07:48 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:48.254162 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d953227-40a9-4964-9d49-4f9f45478d6f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.254555 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d953227-40a9-4964-9d49-4f9f45478d6f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.255034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d953227-40a9-4964-9d49-4f9f45478d6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.269192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186ead8-4f66-4d03-ad91-5c49785e3ec7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=072316d170de4f50a27886d804fa252a, user_id=84a88a406e7349478f31ee93b258eca0, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:48.272133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a186ead8-4f66-4d03-ad91-5c49785e3ec7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:48.286221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-2d953227-40a9-4964-9d49-4f9f45478d6f None None] MFA Rules not processed for user `4751abc9f67947b3ad17ba2b78ce2c12`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:48.288186 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2d953227-40a9-4964-9d49-4f9f45478d6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:48.302578 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 810/1615] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:07:48 2022] PUT /identity/v3/projects/072316d170de4f50a27886d804fa252a/users/84a88a406e7349478f31ee93b258eca0/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:07:48.305027 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186ead8-4f66-4d03-ad91-5c49785e3ec7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:48.420375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a186ead8-4f66-4d03-ad91-5c49785e3ec7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:48.425680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7d3576-b5b1-41a3-92c2-d65385c928d1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:48.427188 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d3576-b5b1-41a3-92c2-d65385c928d1 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.427903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d3576-b5b1-41a3-92c2-d65385c928d1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.428369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7d3576-b5b1-41a3-92c2-d65385c928d1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.435359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 806/1616] 10.222.0.85 () {58 vars in 1058 bytes} [Wed May 25 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:48.437673 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3715796-70d6-4a1a-b9e9-65f5fc41ebce None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.443763 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3715796-70d6-4a1a-b9e9-65f5fc41ebce None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.448696 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3715796-70d6-4a1a-b9e9-65f5fc41ebce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.528059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c3715796-70d6-4a1a-b9e9-65f5fc41ebce None None] MFA Rules not processed for user `4751abc9f67947b3ad17ba2b78ce2c12`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:48.529452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c3715796-70d6-4a1a-b9e9-65f5fc41ebce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:48.553792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d3576-b5b1-41a3-92c2-d65385c928d1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:48.568232 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7d3576-b5b1-41a3-92c2-d65385c928d1 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:48.570722 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 807/1617] 10.222.0.85 () {58 vars in 1058 bytes} [Wed May 25 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:48.571494 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3715796-70d6-4a1a-b9e9-65f5fc41ebce None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:48.582720 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 811/1618] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:07:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 283 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:07:48.584907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21facd80-960e-41ec-89cc-c73f9e0af44d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.585369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21facd80-960e-41ec-89cc-c73f9e0af44d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.586027 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21facd80-960e-41ec-89cc-c73f9e0af44d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.601024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:48.609295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3715796-70d6-4a1a-b9e9-65f5fc41ebce None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:48.621165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60975992-d2c7-4c59-9a32-e9e08e3dc09c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:48.623284 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60975992-d2c7-4c59-9a32-e9e08e3dc09c service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.626884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60975992-d2c7-4c59-9a32-e9e08e3dc09c service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.627318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60975992-d2c7-4c59-9a32-e9e08e3dc09c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.652715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60975992-d2c7-4c59-9a32-e9e08e3dc09c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:48.652867 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-21facd80-960e-41ec-89cc-c73f9e0af44d None None] MFA Rules not processed for user `84a88a406e7349478f31ee93b258eca0`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:48.657038 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60975992-d2c7-4c59-9a32-e9e08e3dc09c service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:48.657766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-21facd80-960e-41ec-89cc-c73f9e0af44d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:48.677909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 808/1619] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:07:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:07:48.678996 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60975992-d2c7-4c59-9a32-e9e08e3dc09c service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:48.703874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60975992-d2c7-4c59-9a32-e9e08e3dc09c service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:48.713370 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff43be9-4b20-42e9-9680-2b152e217eaa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:48.717393 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff43be9-4b20-42e9-9680-2b152e217eaa service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.717948 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff43be9-4b20-42e9-9680-2b152e217eaa service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.718330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff43be9-4b20-42e9-9680-2b152e217eaa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.788382 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 812/1620] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:48 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:48.789459 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21facd80-960e-41ec-89cc-c73f9e0af44d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:48.807573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21facd80-960e-41ec-89cc-c73f9e0af44d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:48.812404 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:48.815518 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c613d-b967-4113-9f1d-10de431397d1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:48.816399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c613d-b967-4113-9f1d-10de431397d1 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.816773 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c613d-b967-4113-9f1d-10de431397d1 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.817162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098c613d-b967-4113-9f1d-10de431397d1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.873930 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff43be9-4b20-42e9-9680-2b152e217eaa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:48.878222 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff43be9-4b20-42e9-9680-2b152e217eaa service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:48.901476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c613d-b967-4113-9f1d-10de431397d1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:48.903242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098c613d-b967-4113-9f1d-10de431397d1 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:48.904525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 809/1621] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:07:48 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:07:48.909007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff43be9-4b20-42e9-9680-2b152e217eaa service nova] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:48.937693 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff43be9-4b20-42e9-9680-2b152e217eaa service nova] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:48.943572 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 813/1622] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:07:48 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 154 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:07:48.956150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6caa8-3593-40f9-ad52-9241b39df9e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:48.957119 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6caa8-3593-40f9-ad52-9241b39df9e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:48.957687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6caa8-3593-40f9-ad52-9241b39df9e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:48.958122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc6caa8-3593-40f9-ad52-9241b39df9e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:48.958978 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6caa8-3593-40f9-ad52-9241b39df9e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:48.961078 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc6caa8-3593-40f9-ad52-9241b39df9e9 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:49.021204 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 810/1623] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:07:48 2022] POST /identity/v3/projects => generated 381 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:49.026313 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c613d-b967-4113-9f1d-10de431397d1 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:49.051811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098c613d-b967-4113-9f1d-10de431397d1 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:49.227859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b582a66-5141-406f-8e06-2e50084113ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:49.228620 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b582a66-5141-406f-8e06-2e50084113ba admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.228914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b582a66-5141-406f-8e06-2e50084113ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.229200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b582a66-5141-406f-8e06-2e50084113ba admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.244566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b582a66-5141-406f-8e06-2e50084113ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:49.246919 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b582a66-5141-406f-8e06-2e50084113ba admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:49.288301 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 814/1624] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:07:49 2022] POST /identity/v3/users => generated 297 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:07:49.292295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6caa8-3593-40f9-ad52-9241b39df9e9 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:49.315162 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc6caa8-3593-40f9-ad52-9241b39df9e9 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:49.325194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f457ab6d-3b21-4f23-af0d-6eabc4898514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:49.325957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457ab6d-3b21-4f23-af0d-6eabc4898514 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.326275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457ab6d-3b21-4f23-af0d-6eabc4898514 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.326612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f457ab6d-3b21-4f23-af0d-6eabc4898514 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.328140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457ab6d-3b21-4f23-af0d-6eabc4898514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:49.336524 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f457ab6d-3b21-4f23-af0d-6eabc4898514 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:49.359575 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 811/1625] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:07:49.361175 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b582a66-5141-406f-8e06-2e50084113ba admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:49.375027 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b582a66-5141-406f-8e06-2e50084113ba admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:49.398127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43527331-970d-4c6b-9d33-b2ba8fd51133 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:49.399390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43527331-970d-4c6b-9d33-b2ba8fd51133 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.399768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43527331-970d-4c6b-9d33-b2ba8fd51133 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.400109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43527331-970d-4c6b-9d33-b2ba8fd51133 admin admin] PATH_INFO: `/v3/projects/5500e045282246458b03e691f0201623/users/06cbac33fe154aff9c1e2e8baf1b37a9/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.438716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43527331-970d-4c6b-9d33-b2ba8fd51133 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5500e045282246458b03e691f0201623, user_id=06cbac33fe154aff9c1e2e8baf1b37a9, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:49.447710 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43527331-970d-4c6b-9d33-b2ba8fd51133 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:49.455464 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5b66a-e43a-47cd-b068-d836050dc40d None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.455753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5b66a-e43a-47cd-b068-d836050dc40d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.456053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db5b66a-e43a-47cd-b068-d836050dc40d None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.457784 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 812/1626] 10.222.0.85 () {54 vars in 912 bytes} [Wed May 25 16:07:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:49.463675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3eb68-d36b-4f21-acb7-67c8567459cc None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.463992 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3eb68-d36b-4f21-acb7-67c8567459cc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.464274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea3eb68-d36b-4f21-acb7-67c8567459cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.481505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:49.497446 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 815/1627] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:07:49 2022] PUT /identity/v3/projects/5500e045282246458b03e691f0201623/users/06cbac33fe154aff9c1e2e8baf1b37a9/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:07:49.498500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43527331-970d-4c6b-9d33-b2ba8fd51133 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:49.503698 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6ea3eb68-d36b-4f21-acb7-67c8567459cc None None] MFA Rules not processed for user `4751abc9f67947b3ad17ba2b78ce2c12`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:49.504985 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6ea3eb68-d36b-4f21-acb7-67c8567459cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:49.639066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43527331-970d-4c6b-9d33-b2ba8fd51133 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:49.651941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d105aa1-841f-441c-9518-62e170f19f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:49.653870 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d105aa1-841f-441c-9518-62e170f19f94 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.655597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d105aa1-841f-441c-9518-62e170f19f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.656114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d105aa1-841f-441c-9518-62e170f19f94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.658223 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d105aa1-841f-441c-9518-62e170f19f94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:49.663664 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d105aa1-841f-441c-9518-62e170f19f94 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:49.668822 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 813/1628] 10.222.0.85 () {58 vars in 1009 bytes} [Wed May 25 16:07:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:07:49.675081 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e06c8-888a-4b10-be21-8568ef6ae1a3 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.675553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e06c8-888a-4b10-be21-8568ef6ae1a3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.675686 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634e06c8-888a-4b10-be21-8568ef6ae1a3 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.679429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 814/1629] 10.222.0.85 () {54 vars in 912 bytes} [Wed May 25 16:07:49 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:49.689076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3531a3f-e05f-46c0-9c0a-7098152887de None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.689191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3531a3f-e05f-46c0-9c0a-7098152887de None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.689526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3531a3f-e05f-46c0-9c0a-7098152887de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.692290 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 816/1630] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:07:49 2022] GET /identity/v3/roles => generated 1420 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:07:49.693251 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d105aa1-841f-441c-9518-62e170f19f94 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:49.714892 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d105aa1-841f-441c-9518-62e170f19f94 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:49.726380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce64b3-f636-4615-86bd-9df3981cc059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:49.726776 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f3531a3f-e05f-46c0-9c0a-7098152887de None None] MFA Rules not processed for user `4751abc9f67947b3ad17ba2b78ce2c12`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:49.729363 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f3531a3f-e05f-46c0-9c0a-7098152887de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:49.729558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce64b3-f636-4615-86bd-9df3981cc059 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.729780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce64b3-f636-4615-86bd-9df3981cc059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.730063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ce64b3-f636-4615-86bd-9df3981cc059 admin admin] PATH_INFO: `/v3/projects/5500e045282246458b03e691f0201623/users/06cbac33fe154aff9c1e2e8baf1b37a9/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.734485 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce64b3-f636-4615-86bd-9df3981cc059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5500e045282246458b03e691f0201623, user_id=06cbac33fe154aff9c1e2e8baf1b37a9, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:49.740957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ce64b3-f636-4615-86bd-9df3981cc059 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:49.751927 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 815/1631] 10.222.0.85 () {58 vars in 1009 bytes} [Wed May 25 16:07:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:49.753603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3531a3f-e05f-46c0-9c0a-7098152887de None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:49.772387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 817/1632] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:07:49 2022] PUT /identity/v3/projects/5500e045282246458b03e691f0201623/users/06cbac33fe154aff9c1e2e8baf1b37a9/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:07:49.777490 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce64b3-f636-4615-86bd-9df3981cc059 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:49.903499 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3531a3f-e05f-46c0-9c0a-7098152887de None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:49.911649 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-256184e2-eef7-4e67-90de-6747f3e3bd5e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:49.913148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256184e2-eef7-4e67-90de-6747f3e3bd5e service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.915429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256184e2-eef7-4e67-90de-6747f3e3bd5e service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.916494 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-256184e2-eef7-4e67-90de-6747f3e3bd5e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:49.918066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ce64b3-f636-4615-86bd-9df3981cc059 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:49.928162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b51bae-7c1f-45d8-b232-d6034698c627 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:49.930972 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b51bae-7c1f-45d8-b232-d6034698c627 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:49.932028 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b51bae-7c1f-45d8-b232-d6034698c627 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:49.932923 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b51bae-7c1f-45d8-b232-d6034698c627 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:50.012476 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:50.087486 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256184e2-eef7-4e67-90de-6747f3e3bd5e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:50.091369 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-256184e2-eef7-4e67-90de-6747f3e3bd5e service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:50.104997 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b51bae-7c1f-45d8-b232-d6034698c627 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:50.107682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7b51bae-7c1f-45d8-b232-d6034698c627 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:50.120317 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 816/1633] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:07:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:07:50.126074 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99a09e-ac1b-4e11-a5e0-b6d93c0ea16f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:50.128209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99a09e-ac1b-4e11-a5e0-b6d93c0ea16f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:50.128209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c99a09e-ac1b-4e11-a5e0-b6d93c0ea16f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:50.137218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 818/1634] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:07:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:07:50.166828 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8c99a09e-ac1b-4e11-a5e0-b6d93c0ea16f None None] MFA Rules not processed for user `06cbac33fe154aff9c1e2e8baf1b37a9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:50.168641 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8c99a09e-ac1b-4e11-a5e0-b6d93c0ea16f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:50.294193 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 817/1635] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:50 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:50.902276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b51bae-7c1f-45d8-b232-d6034698c627 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:50.912860 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7b51bae-7c1f-45d8-b232-d6034698c627 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:50.922828 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6fe5a12-3a64-4eed-8cbc-ffa87ac6c6a1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:50.923517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe5a12-3a64-4eed-8cbc-ffa87ac6c6a1 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:50.923813 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe5a12-3a64-4eed-8cbc-ffa87ac6c6a1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:50.924530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6fe5a12-3a64-4eed-8cbc-ffa87ac6c6a1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:50.963245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe5a12-3a64-4eed-8cbc-ffa87ac6c6a1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:50.964559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6fe5a12-3a64-4eed-8cbc-ffa87ac6c6a1 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:50.995107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 819/1636] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:07:50 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:07:51.364625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99a09e-ac1b-4e11-a5e0-b6d93c0ea16f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:51.377104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c99a09e-ac1b-4e11-a5e0-b6d93c0ea16f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:51.383357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1f7c8-06b7-441a-a684-9216c4024fc8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:51.384419 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1f7c8-06b7-441a-a684-9216c4024fc8 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:51.384844 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1f7c8-06b7-441a-a684-9216c4024fc8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:51.385297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1f7c8-06b7-441a-a684-9216c4024fc8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:51.520931 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1f7c8-06b7-441a-a684-9216c4024fc8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:51.522583 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1f7c8-06b7-441a-a684-9216c4024fc8 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:51.536885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 818/1637] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:07:51 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:07:52.161248 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cf67f-312f-45fd-9818-0c9818c5de9e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:52.164711 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cf67f-312f-45fd-9818-0c9818c5de9e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:52.165270 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-723cf67f-312f-45fd-9818-0c9818c5de9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:52.200675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:52.220417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-723cf67f-312f-45fd-9818-0c9818c5de9e None None] MFA Rules not processed for user `1d4d043b13da4b17830481f4a8da9b79`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:52.223536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-723cf67f-312f-45fd-9818-0c9818c5de9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:52.366188 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 820/1638] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:52.378048 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b9a66-d2f4-4ab6-b478-f6a2a91cbd1a None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:52.378599 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b9a66-d2f4-4ab6-b478-f6a2a91cbd1a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:52.378599 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179b9a66-d2f4-4ab6-b478-f6a2a91cbd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:52.427698 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-179b9a66-d2f4-4ab6-b478-f6a2a91cbd1a None None] MFA Rules not processed for user `1d4d043b13da4b17830481f4a8da9b79`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:52.428981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-179b9a66-d2f4-4ab6-b478-f6a2a91cbd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:52.443963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 819/1639] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 67 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:52.455199 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cf67f-312f-45fd-9818-0c9818c5de9e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:52.471338 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-723cf67f-312f-45fd-9818-0c9818c5de9e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:52.488549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261c2eea-75a6-4b5d-ba38-6110233049fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:52.489455 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c2eea-75a6-4b5d-ba38-6110233049fc service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:52.489921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c2eea-75a6-4b5d-ba38-6110233049fc service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:52.490357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261c2eea-75a6-4b5d-ba38-6110233049fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:52.515266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c2eea-75a6-4b5d-ba38-6110233049fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:52.520116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261c2eea-75a6-4b5d-ba38-6110233049fc service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:52.556312 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 821/1640] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:07:52 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:07:53.385711 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3374a26-a47d-4612-a995-8b6b5748bcbf None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:53.386226 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3374a26-a47d-4612-a995-8b6b5748bcbf None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:53.386316 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3374a26-a47d-4612-a995-8b6b5748bcbf None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:53.387768 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 820/1641] 10.222.0.85 () {54 vars in 912 bytes} [Wed May 25 16:07:53 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:53.408144 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e3b20-5414-494f-991a-e7fc054852f7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:53.408654 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e3b20-5414-494f-991a-e7fc054852f7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:53.408723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757e3b20-5414-494f-991a-e7fc054852f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:53.488990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-757e3b20-5414-494f-991a-e7fc054852f7 None None] MFA Rules not processed for user `413b3d65427b41dc99bc7b0a94fb1dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:53.490260 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-757e3b20-5414-494f-991a-e7fc054852f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:53.623907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 822/1642] 10.222.0.85 () {58 vars in 1009 bytes} [Wed May 25 16:07:53 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:53.660086 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3374a26-a47d-4612-a995-8b6b5748bcbf None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:53.682712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3374a26-a47d-4612-a995-8b6b5748bcbf None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:53.691286 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05357d05-518d-487f-8b69-56e8feb1f3d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:53.692877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05357d05-518d-487f-8b69-56e8feb1f3d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:53.693278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05357d05-518d-487f-8b69-56e8feb1f3d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:53.694569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05357d05-518d-487f-8b69-56e8feb1f3d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:53.713260 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05357d05-518d-487f-8b69-56e8feb1f3d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:53.719016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05357d05-518d-487f-8b69-56e8feb1f3d8 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:53.747328 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 821/1643] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:07:53 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:07:55.916970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7116f7-28da-4ce2-a1dd-5637d2d46b2b None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:55.916970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7116f7-28da-4ce2-a1dd-5637d2d46b2b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:55.916970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7116f7-28da-4ce2-a1dd-5637d2d46b2b None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:55.918839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 823/1644] 10.222.0.85 () {54 vars in 960 bytes} [Wed May 25 16:07:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:07:55.922694 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904aca6-c949-436a-b7ac-4aa185bed05d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:55.923151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904aca6-c949-436a-b7ac-4aa185bed05d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:55.923258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1904aca6-c949-436a-b7ac-4aa185bed05d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:55.957101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-1904aca6-c949-436a-b7ac-4aa185bed05d None None] MFA Rules not processed for user `82b738f1b8c64590955a640c461b4ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:55.959564 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1904aca6-c949-436a-b7ac-4aa185bed05d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:56.084991 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:56.164875 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7116f7-28da-4ce2-a1dd-5637d2d46b2b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:56.178726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 822/1645] 10.222.0.85 () {58 vars in 1057 bytes} [Wed May 25 16:07:55 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:56.187428 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7116f7-28da-4ce2-a1dd-5637d2d46b2b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:56.194676 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13429607-8994-4a54-9a99-25341d2376d2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b738f1b8c64590955a640c461b4ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:56.195574 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13429607-8994-4a54-9a99-25341d2376d2 service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:56.196057 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13429607-8994-4a54-9a99-25341d2376d2 service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:56.196477 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13429607-8994-4a54-9a99-25341d2376d2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:56.200601 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-13429607-8994-4a54-9a99-25341d2376d2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:56.224469 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13429607-8994-4a54-9a99-25341d2376d2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:56.226181 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13429607-8994-4a54-9a99-25341d2376d2 service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:56.252490 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 824/1646] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:07:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:07:57.131614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904aca6-c949-436a-b7ac-4aa185bed05d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:57.373154 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1904aca6-c949-436a-b7ac-4aa185bed05d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:57.401418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b5172-0b91-411e-bb3d-05a3acc56d23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:57.405084 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b5172-0b91-411e-bb3d-05a3acc56d23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:57.405524 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b5172-0b91-411e-bb3d-05a3acc56d23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:57.405971 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34b5172-0b91-411e-bb3d-05a3acc56d23 admin admin] PATH_INFO: `/v3/users/09ea4058ec5f4cbea82069ee235ef911` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:57.409763 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b5172-0b91-411e-bb3d-05a3acc56d23 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09ea4058ec5f4cbea82069ee235ef911)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:57.424419 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34b5172-0b91-411e-bb3d-05a3acc56d23 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:57.580944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b34b5172-0b91-411e-bb3d-05a3acc56d23 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09ea4058ec5f4cbea82069ee235ef911'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:07:57.581854 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b34b5172-0b91-411e-bb3d-05a3acc56d23 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09ea4058ec5f4cbea82069ee235ef911'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:07:57.594818 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b34b5172-0b91-411e-bb3d-05a3acc56d23 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09ea4058ec5f4cbea82069ee235ef911'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:07:57.617269 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b34b5172-0b91-411e-bb3d-05a3acc56d23 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09ea4058ec5f4cbea82069ee235ef911'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:07:57.618074 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b34b5172-0b91-411e-bb3d-05a3acc56d23 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09ea4058ec5f4cbea82069ee235ef911'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:07:57.639365 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 823/1647] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:07:57 2022] DELETE /identity/v3/users/09ea4058ec5f4cbea82069ee235ef911 => generated 0 bytes in 508 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:07:57.651587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854a3b7-f0b0-405c-a435-fcb2f5569044 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:57.652468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854a3b7-f0b0-405c-a435-fcb2f5569044 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:57.652897 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5854a3b7-f0b0-405c-a435-fcb2f5569044 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:57.711587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5854a3b7-f0b0-405c-a435-fcb2f5569044 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:57.713957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5854a3b7-f0b0-405c-a435-fcb2f5569044 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:57.901135 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 825/1648] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:57.943734 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b5172-0b91-411e-bb3d-05a3acc56d23 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:57.946970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b34b5172-0b91-411e-bb3d-05a3acc56d23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:58.104839 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:58.137009 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34b5172-0b91-411e-bb3d-05a3acc56d23 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:58.156850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c192a89-45a2-4f6f-807c-97fdddc4583a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:58.157797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c192a89-45a2-4f6f-807c-97fdddc4583a service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:58.158084 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c192a89-45a2-4f6f-807c-97fdddc4583a service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:58.158363 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c192a89-45a2-4f6f-807c-97fdddc4583a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:58.194728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c192a89-45a2-4f6f-807c-97fdddc4583a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:58.196306 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c192a89-45a2-4f6f-807c-97fdddc4583a service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:58.208667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 824/1649] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:07:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 266 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:07:59.468160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb0d7b-bb9f-4bfa-83b2-e833bc25bb16 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:59.469109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb0d7b-bb9f-4bfa-83b2-e833bc25bb16 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:59.469604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5eb0d7b-bb9f-4bfa-83b2-e833bc25bb16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:59.502006 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:07:59.521508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c5eb0d7b-bb9f-4bfa-83b2-e833bc25bb16 None None] MFA Rules not processed for user `d0427d9cce40423185a40df61c3617a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:59.526069 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c5eb0d7b-bb9f-4bfa-83b2-e833bc25bb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:59.630341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 826/1650] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:59.643300 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfa30f-df7f-4822-8339-526798924735 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:59.644318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfa30f-df7f-4822-8339-526798924735 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:59.645082 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80dfa30f-df7f-4822-8339-526798924735 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:59.694399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-80dfa30f-df7f-4822-8339-526798924735 None None] MFA Rules not processed for user `d0427d9cce40423185a40df61c3617a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:59.696031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-80dfa30f-df7f-4822-8339-526798924735 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:59.719755 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 825/1651] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 73 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:07:59.726366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb0d7b-bb9f-4bfa-83b2-e833bc25bb16 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:07:59.728129 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c5eb0d7b-bb9f-4bfa-83b2-e833bc25bb16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:59.809196 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e67551-3564-4309-bdb4-1dd8f7cc8316 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:59.809486 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e67551-3564-4309-bdb4-1dd8f7cc8316 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:59.809764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e67551-3564-4309-bdb4-1dd8f7cc8316 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:59.874547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-89e67551-3564-4309-bdb4-1dd8f7cc8316 None None] MFA Rules not processed for user `be5c51fdd2fe4159a83a560dc074da6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:07:59.875771 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-89e67551-3564-4309-bdb4-1dd8f7cc8316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:07:59.893262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5eb0d7b-bb9f-4bfa-83b2-e833bc25bb16 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:07:59.907063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aebadb-2d46-4937-afe6-56120de61fcc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:07:59.907906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aebadb-2d46-4937-afe6-56120de61fcc service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:07:59.910843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aebadb-2d46-4937-afe6-56120de61fcc service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:07:59.911258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8aebadb-2d46-4937-afe6-56120de61fcc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:07:59.935366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aebadb-2d46-4937-afe6-56120de61fcc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:07:59.963852 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8aebadb-2d46-4937-afe6-56120de61fcc service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:07:59.987805 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 827/1652] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:07:59 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 262 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:00.052317 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 826/1653] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:07:59 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:00.084451 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aebadb-2d46-4937-afe6-56120de61fcc service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:00.086712 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e8aebadb-2d46-4937-afe6-56120de61fcc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:00.120065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e67551-3564-4309-bdb4-1dd8f7cc8316 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:00.121525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-89e67551-3564-4309-bdb4-1dd8f7cc8316 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:00.138667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8aebadb-2d46-4937-afe6-56120de61fcc service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:00.157075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecabda51-14b0-46e0-a342-84f3f34bf00b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:00.157820 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabda51-14b0-46e0-a342-84f3f34bf00b service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:00.159751 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabda51-14b0-46e0-a342-84f3f34bf00b service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:00.160294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecabda51-14b0-46e0-a342-84f3f34bf00b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:00.163157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e67551-3564-4309-bdb4-1dd8f7cc8316 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:00.181873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c3a3c5-36d5-46f6-b8cf-5029a6265280 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:00.182987 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3a3c5-36d5-46f6-b8cf-5029a6265280 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:00.183416 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3a3c5-36d5-46f6-b8cf-5029a6265280 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:00.184043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c3a3c5-36d5-46f6-b8cf-5029a6265280 admin admin] PATH_INFO: `/v3/projects/0b10cd68403a4ef7888e6ed340a04916` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:00.186662 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3a3c5-36d5-46f6-b8cf-5029a6265280 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0b10cd68403a4ef7888e6ed340a04916)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:00.187797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: WARNING py.warnings [None req-53c3a3c5-36d5-46f6-b8cf-5029a6265280 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:08:00.187797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: warnings.warn(msg) May 25 16:08:00.187797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: May 25 16:08:00.189024 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c3a3c5-36d5-46f6-b8cf-5029a6265280 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:00.201124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabda51-14b0-46e0-a342-84f3f34bf00b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:00.202782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecabda51-14b0-46e0-a342-84f3f34bf00b service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:00.238077 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 828/1654] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:00 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:00.343597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-53c3a3c5-36d5-46f6-b8cf-5029a6265280 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0b10cd68403a4ef7888e6ed340a04916', 'request_id': 'req-53c3a3c5-36d5-46f6-b8cf-5029a6265280'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:00.352464 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-53c3a3c5-36d5-46f6-b8cf-5029a6265280 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0b10cd68403a4ef7888e6ed340a04916', 'request_id': 'req-53c3a3c5-36d5-46f6-b8cf-5029a6265280'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:00.377288 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-53c3a3c5-36d5-46f6-b8cf-5029a6265280 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0b10cd68403a4ef7888e6ed340a04916', 'request_id': 'req-53c3a3c5-36d5-46f6-b8cf-5029a6265280'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:00.448791 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-53c3a3c5-36d5-46f6-b8cf-5029a6265280 admin admin] The token cache is being invalidate because project 0b10cd68403a4ef7888e6ed340a04916 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:00.449364 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-53c3a3c5-36d5-46f6-b8cf-5029a6265280 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:00.462308 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-53c3a3c5-36d5-46f6-b8cf-5029a6265280 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:00.468840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 827/1655] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:00 2022] DELETE /identity/v3/projects/0b10cd68403a4ef7888e6ed340a04916 => generated 0 bytes in 346 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:00.470871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeed7c-bb44-473b-8da9-5f47ba313d8d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:00.471340 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeed7c-bb44-473b-8da9-5f47ba313d8d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:00.471520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbeed7c-bb44-473b-8da9-5f47ba313d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:00.523538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4fbeed7c-bb44-473b-8da9-5f47ba313d8d None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:00.524789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4fbeed7c-bb44-473b-8da9-5f47ba313d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:00.694036 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 829/1656] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:00.703056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a10f7-6b03-4831-906e-e3c28f6244d0 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:00.704914 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a10f7-6b03-4831-906e-e3c28f6244d0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:00.705389 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-835a10f7-6b03-4831-906e-e3c28f6244d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:00.753987 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-835a10f7-6b03-4831-906e-e3c28f6244d0 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:00.762446 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-835a10f7-6b03-4831-906e-e3c28f6244d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:00.809155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 828/1657] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:00.813710 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbeed7c-bb44-473b-8da9-5f47ba313d8d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:00.837334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbeed7c-bb44-473b-8da9-5f47ba313d8d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:00.855156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e730c22-48f6-479a-9728-46dc3441cdc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:00.856259 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e730c22-48f6-479a-9728-46dc3441cdc9 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:00.856725 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e730c22-48f6-479a-9728-46dc3441cdc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:00.857173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e730c22-48f6-479a-9728-46dc3441cdc9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:00.858269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e730c22-48f6-479a-9728-46dc3441cdc9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:00.858872 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e730c22-48f6-479a-9728-46dc3441cdc9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:00.861211 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e730c22-48f6-479a-9728-46dc3441cdc9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:00.870075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:00.884379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a10f7-6b03-4831-906e-e3c28f6244d0 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:00.890479 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 830/1658] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:00.896960 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-835a10f7-6b03-4831-906e-e3c28f6244d0 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:00.913326 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2efaef-3cce-45a8-a60b-71b53fed573b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:00.914109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2efaef-3cce-45a8-a60b-71b53fed573b admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:00.914463 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2efaef-3cce-45a8-a60b-71b53fed573b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:00.914854 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a2efaef-3cce-45a8-a60b-71b53fed573b admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:00.915782 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2efaef-3cce-45a8-a60b-71b53fed573b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:00.917894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a2efaef-3cce-45a8-a60b-71b53fed573b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:00.985702 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 829/1659] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:00 2022] POST /identity/v3/projects => generated 365 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:00.987008 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e730c22-48f6-479a-9728-46dc3441cdc9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:01.005877 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e730c22-48f6-479a-9728-46dc3441cdc9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:01.191360 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9465e0-1e74-443b-a44b-e31b2f2561cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:01.192504 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9465e0-1e74-443b-a44b-e31b2f2561cb admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:01.193016 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9465e0-1e74-443b-a44b-e31b2f2561cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:01.193510 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9465e0-1e74-443b-a44b-e31b2f2561cb admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:01.194424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9465e0-1e74-443b-a44b-e31b2f2561cb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:01.196912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9465e0-1e74-443b-a44b-e31b2f2561cb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:01.266865 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 831/1660] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:00 2022] POST /identity/v3/users => generated 283 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:01.267758 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2efaef-3cce-45a8-a60b-71b53fed573b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:01.313310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a2efaef-3cce-45a8-a60b-71b53fed573b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:01.337394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e754af-edf3-42d2-b49d-c4910786f106 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:01.344870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e754af-edf3-42d2-b49d-c4910786f106 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:01.347044 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e754af-edf3-42d2-b49d-c4910786f106 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:01.347453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e754af-edf3-42d2-b49d-c4910786f106 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:01.361252 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e754af-edf3-42d2-b49d-c4910786f106 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:01.373887 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e754af-edf3-42d2-b49d-c4910786f106 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:01.493675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 830/1661] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:01.496799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9465e0-1e74-443b-a44b-e31b2f2561cb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:01.515937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9465e0-1e74-443b-a44b-e31b2f2561cb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:01.536545 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0223b32e-e9f3-448d-b3dc-7250778bfbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:01.537424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223b32e-e9f3-448d-b3dc-7250778bfbe5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:01.537855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223b32e-e9f3-448d-b3dc-7250778bfbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:01.538840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0223b32e-e9f3-448d-b3dc-7250778bfbe5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:01.543078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223b32e-e9f3-448d-b3dc-7250778bfbe5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:01.544927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0223b32e-e9f3-448d-b3dc-7250778bfbe5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:01.574354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e754af-edf3-42d2-b49d-c4910786f106 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:01.584220 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 832/1662] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:01 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:01.589444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e754af-edf3-42d2-b49d-c4910786f106 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:01.607587 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b3824-2758-49e5-a305-07d748c000ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:01.608776 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b3824-2758-49e5-a305-07d748c000ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:01.611701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b3824-2758-49e5-a305-07d748c000ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:01.612182 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-463b3824-2758-49e5-a305-07d748c000ad admin admin] PATH_INFO: `/v3/projects/684290bd47864be38ec9736193b7e1ce/users/5d8147e581634e718814eb52886bcbb3/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:01.685114 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:01.695589 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b3824-2758-49e5-a305-07d748c000ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=684290bd47864be38ec9736193b7e1ce, user_id=5d8147e581634e718814eb52886bcbb3, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:01.700065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-463b3824-2758-49e5-a305-07d748c000ad admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:01.752362 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 831/1663] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:01 2022] PUT /identity/v3/projects/684290bd47864be38ec9736193b7e1ce/users/5d8147e581634e718814eb52886bcbb3/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:01.764211 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154520ba-9671-4737-b154-5010918a6944 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:01.767438 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154520ba-9671-4737-b154-5010918a6944 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:01.768698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-154520ba-9671-4737-b154-5010918a6944 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:01.883155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b3824-2758-49e5-a305-07d748c000ad admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:01.887251 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-463b3824-2758-49e5-a305-07d748c000ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:01.936604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-154520ba-9671-4737-b154-5010918a6944 None None] MFA Rules not processed for user `5d8147e581634e718814eb52886bcbb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:01.938126 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-154520ba-9671-4737-b154-5010918a6944 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:02.131021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 833/1664] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:02.132471 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154520ba-9671-4737-b154-5010918a6944 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:02.142214 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-463b3824-2758-49e5-a305-07d748c000ad admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:02.154508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad3a7e1-bbf1-45eb-8ef7-585f2722f582 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:02.156121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3a7e1-bbf1-45eb-8ef7-585f2722f582 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:02.156431 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3a7e1-bbf1-45eb-8ef7-585f2722f582 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:02.156715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad3a7e1-bbf1-45eb-8ef7-585f2722f582 admin admin] PATH_INFO: `/v3/users/e970b42de0dd4ecbb11bfa8401d234e5` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:02.159572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3a7e1-bbf1-45eb-8ef7-585f2722f582 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e970b42de0dd4ecbb11bfa8401d234e5)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:02.168431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-154520ba-9671-4737-b154-5010918a6944 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:02.170151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad3a7e1-bbf1-45eb-8ef7-585f2722f582 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:02.180478 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e34b2-3155-4a33-b5cf-618f69408959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:02.183099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e34b2-3155-4a33-b5cf-618f69408959 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:02.184335 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e34b2-3155-4a33-b5cf-618f69408959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:02.185522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3e34b2-3155-4a33-b5cf-618f69408959 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:02.188489 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e34b2-3155-4a33-b5cf-618f69408959 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:02.196542 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3e34b2-3155-4a33-b5cf-618f69408959 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:02.331338 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 834/1665] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:02 2022] POST /identity/v3/projects => generated 363 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:02.332340 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e34b2-3155-4a33-b5cf-618f69408959 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:02.353978 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7ad3a7e1-bbf1-45eb-8ef7-585f2722f582 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e970b42de0dd4ecbb11bfa8401d234e5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:02.356647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7ad3a7e1-bbf1-45eb-8ef7-585f2722f582 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e970b42de0dd4ecbb11bfa8401d234e5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:02.396648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7ad3a7e1-bbf1-45eb-8ef7-585f2722f582 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e970b42de0dd4ecbb11bfa8401d234e5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:02.416813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7ad3a7e1-bbf1-45eb-8ef7-585f2722f582 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e970b42de0dd4ecbb11bfa8401d234e5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:02.417694 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7ad3a7e1-bbf1-45eb-8ef7-585f2722f582 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e970b42de0dd4ecbb11bfa8401d234e5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:02.469785 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 832/1666] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:01 2022] DELETE /identity/v3/users/e970b42de0dd4ecbb11bfa8401d234e5 => generated 0 bytes in 587 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:02.485655 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e04697-607f-459e-be22-68a79664d78e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:02.486297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e04697-607f-459e-be22-68a79664d78e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:02.486658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e04697-607f-459e-be22-68a79664d78e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:02.490012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:02.546958 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3e34b2-3155-4a33-b5cf-618f69408959 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:02.556241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f7e04697-607f-459e-be22-68a79664d78e None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:02.566270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f7e04697-607f-459e-be22-68a79664d78e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:02.812481 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9eb4f-d0ff-4c98-b95f-60e5ba3606b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:02.815761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9eb4f-d0ff-4c98-b95f-60e5ba3606b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:02.816221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9eb4f-d0ff-4c98-b95f-60e5ba3606b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:02.817800 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a9eb4f-d0ff-4c98-b95f-60e5ba3606b3 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:02.817800 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9eb4f-d0ff-4c98-b95f-60e5ba3606b3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:02.819895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a9eb4f-d0ff-4c98-b95f-60e5ba3606b3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:02.822338 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 833/1667] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:02.823596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe10a1-9ddc-403f-a635-7d994f70f438 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:02.823910 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe10a1-9ddc-403f-a635-7d994f70f438 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:02.824193 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fe10a1-9ddc-403f-a635-7d994f70f438 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:02.841610 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 834/1668] 10.222.0.85 () {54 vars in 912 bytes} [Wed May 25 16:08:02 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:02.845114 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850254c6-2500-49eb-a6fd-43c243810198 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:02.853283 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850254c6-2500-49eb-a6fd-43c243810198 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:02.853937 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850254c6-2500-49eb-a6fd-43c243810198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:02.870029 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 835/1669] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:02 2022] POST /identity/v3/users => generated 288 bytes in 538 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:02.873866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9eb4f-d0ff-4c98-b95f-60e5ba3606b3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:02.885976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-63a9eb4f-d0ff-4c98-b95f-60e5ba3606b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:02.914432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a9eb4f-d0ff-4c98-b95f-60e5ba3606b3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:02.922553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-850254c6-2500-49eb-a6fd-43c243810198 None None] MFA Rules not processed for user `413b3d65427b41dc99bc7b0a94fb1dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:02.925655 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-850254c6-2500-49eb-a6fd-43c243810198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:02.946585 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a6431-a1e7-42e5-8c91-c2b847bc3838 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:02.952147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a6431-a1e7-42e5-8c91-c2b847bc3838 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:02.954826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a6431-a1e7-42e5-8c91-c2b847bc3838 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:02.955332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5a6431-a1e7-42e5-8c91-c2b847bc3838 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:02.955507 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:02.956977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a6431-a1e7-42e5-8c91-c2b847bc3838 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:02.965984 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5a6431-a1e7-42e5-8c91-c2b847bc3838 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:03.008209 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 836/1670] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:02 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:03.009233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a6431-a1e7-42e5-8c91-c2b847bc3838 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:03.014392 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cb5a6431-a1e7-42e5-8c91-c2b847bc3838 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:03.175193 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5a6431-a1e7-42e5-8c91-c2b847bc3838 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:03.180245 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 835/1671] 10.222.0.85 () {58 vars in 1009 bytes} [Wed May 25 16:08:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:03.181220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850254c6-2500-49eb-a6fd-43c243810198 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:03.193641 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8443fb43-f4cb-4070-ada8-d5105a2dcd3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:03.194351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443fb43-f4cb-4070-ada8-d5105a2dcd3f service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:03.194647 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443fb43-f4cb-4070-ada8-d5105a2dcd3f service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:03.194929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8443fb43-f4cb-4070-ada8-d5105a2dcd3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:03.224795 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443fb43-f4cb-4070-ada8-d5105a2dcd3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:03.226313 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8443fb43-f4cb-4070-ada8-d5105a2dcd3f service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:03.227020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850254c6-2500-49eb-a6fd-43c243810198 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:03.248316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 837/1672] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:03.253391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f09067-6620-49db-90e0-8531940090a9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:03.253698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f09067-6620-49db-90e0-8531940090a9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:03.253984 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19f09067-6620-49db-90e0-8531940090a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:03.260823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60157970-b566-492f-9993-f2aff4ef7b43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:03.261776 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60157970-b566-492f-9993-f2aff4ef7b43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:03.262069 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60157970-b566-492f-9993-f2aff4ef7b43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:03.262353 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60157970-b566-492f-9993-f2aff4ef7b43 admin admin] PATH_INFO: `/v3/projects/3bd7991d000c47b794ff72c5be832231/users/7765d0e3cb5e404199840c867260c1b1/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:03.278870 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-19f09067-6620-49db-90e0-8531940090a9 None None] MFA Rules not processed for user `066de1f11f724eee9df0377beb58d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:03.281209 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-19f09067-6620-49db-90e0-8531940090a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:03.320329 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60157970-b566-492f-9993-f2aff4ef7b43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd7991d000c47b794ff72c5be832231, user_id=7765d0e3cb5e404199840c867260c1b1, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:03.327033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60157970-b566-492f-9993-f2aff4ef7b43 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:03.356476 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 836/1673] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:03 2022] PUT /identity/v3/projects/3bd7991d000c47b794ff72c5be832231/users/7765d0e3cb5e404199840c867260c1b1/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:03.360293 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60157970-b566-492f-9993-f2aff4ef7b43 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:03.449677 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 838/1674] 10.222.0.85 () {58 vars in 1009 bytes} [Wed May 25 16:08:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:03.474759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f09067-6620-49db-90e0-8531940090a9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:03.489585 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19f09067-6620-49db-90e0-8531940090a9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:03.527369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:03.534548 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60157970-b566-492f-9993-f2aff4ef7b43 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:03.546893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437f9a3-95e6-45d4-97c1-ab8b1f29f583 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:03.547617 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437f9a3-95e6-45d4-97c1-ab8b1f29f583 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:03.547952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437f9a3-95e6-45d4-97c1-ab8b1f29f583 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:03.548237 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4437f9a3-95e6-45d4-97c1-ab8b1f29f583 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:03.550203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4437f9a3-95e6-45d4-97c1-ab8b1f29f583 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:03.553052 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4437f9a3-95e6-45d4-97c1-ab8b1f29f583 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:03.579144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 837/1675] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:03 2022] GET /identity/v3/roles => generated 1420 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:03.583217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437f9a3-95e6-45d4-97c1-ab8b1f29f583 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:03.606972 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6bbbb-1290-417f-9ca4-2f78a4f8a4cd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:03.607693 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6bbbb-1290-417f-9ca4-2f78a4f8a4cd service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:03.608025 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6bbbb-1290-417f-9ca4-2f78a4f8a4cd service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:03.608354 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a6bbbb-1290-417f-9ca4-2f78a4f8a4cd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:03.620075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a6bbbb-1290-417f-9ca4-2f78a4f8a4cd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:03.622697 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a6bbbb-1290-417f-9ca4-2f78a4f8a4cd service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:03.632036 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4437f9a3-95e6-45d4-97c1-ab8b1f29f583 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:03.636818 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 839/1676] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:03.661939 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0abcc0f-e355-48bb-ae3b-02ade4e7cd3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:03.662881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abcc0f-e355-48bb-ae3b-02ade4e7cd3a admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:03.663168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abcc0f-e355-48bb-ae3b-02ade4e7cd3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:03.663457 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0abcc0f-e355-48bb-ae3b-02ade4e7cd3a admin admin] PATH_INFO: `/v3/projects/3bd7991d000c47b794ff72c5be832231/users/7765d0e3cb5e404199840c867260c1b1/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:03.665678 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0abcc0f-e355-48bb-ae3b-02ade4e7cd3a admin admin] RBAC: Authorizing `identity:create_grant(project_id=3bd7991d000c47b794ff72c5be832231, user_id=7765d0e3cb5e404199840c867260c1b1, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:03.668444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0abcc0f-e355-48bb-ae3b-02ade4e7cd3a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:03.697332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6bbbb-1290-417f-9ca4-2f78a4f8a4cd service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:03.699850 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-96a6bbbb-1290-417f-9ca4-2f78a4f8a4cd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:03.703240 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 838/1677] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:03 2022] PUT /identity/v3/projects/3bd7991d000c47b794ff72c5be832231/users/7765d0e3cb5e404199840c867260c1b1/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:03.715500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadabe7-69c7-40d7-bd71-0be9b8140c06 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:03.715859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadabe7-69c7-40d7-bd71-0be9b8140c06 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:03.716148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eadabe7-69c7-40d7-bd71-0be9b8140c06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:03.809721 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-0eadabe7-69c7-40d7-bd71-0be9b8140c06 None None] MFA Rules not processed for user `7765d0e3cb5e404199840c867260c1b1`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:03.810978 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0eadabe7-69c7-40d7-bd71-0be9b8140c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:03.842463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a6bbbb-1290-417f-9ca4-2f78a4f8a4cd service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:04.019874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 839/1678] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:03 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 305 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:04.034420 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadabe7-69c7-40d7-bd71-0be9b8140c06 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:04.037443 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0eadabe7-69c7-40d7-bd71-0be9b8140c06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:04.040064 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd64b8-24f3-4715-9519-aba19a5cc448 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77fda27641c84fccb68a5cdaf4e8dc4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:04.040904 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd64b8-24f3-4715-9519-aba19a5cc448 service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:04.041205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd64b8-24f3-4715-9519-aba19a5cc448 service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:04.041497 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd64b8-24f3-4715-9519-aba19a5cc448 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:04.061347 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:04.179057 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd64b8-24f3-4715-9519-aba19a5cc448 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:04.181322 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fd64b8-24f3-4715-9519-aba19a5cc448 service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:04.201232 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 840/1679] 10.222.0.85 () {60 vars in 1384 bytes} [Wed May 25 16:08:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 505 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 16:08:04.220113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eadabe7-69c7-40d7-bd71-0be9b8140c06 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:04.227660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b21c5af2-a05a-4706-bdf7-313b958290a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:04.228251 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c5af2-a05a-4706-bdf7-313b958290a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:04.228545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c5af2-a05a-4706-bdf7-313b958290a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:04.228830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b21c5af2-a05a-4706-bdf7-313b958290a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:04.269529 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c5af2-a05a-4706-bdf7-313b958290a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:04.279837 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b21c5af2-a05a-4706-bdf7-313b958290a5 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:04.343459 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 840/1680] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:04 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:04.591659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd64b8-24f3-4715-9519-aba19a5cc448 service swift] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:04.598407 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:04.608692 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd64b8-24f3-4715-9519-aba19a5cc448 service swift] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:04.626841 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385204c-de73-483b-9eb0-034c42d008e2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:04.627621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385204c-de73-483b-9eb0-034c42d008e2 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:04.633796 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385204c-de73-483b-9eb0-034c42d008e2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:04.633796 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f385204c-de73-483b-9eb0-034c42d008e2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:04.645659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f385204c-de73-483b-9eb0-034c42d008e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:04.951266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385204c-de73-483b-9eb0-034c42d008e2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:04.952983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f385204c-de73-483b-9eb0-034c42d008e2 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:04.985770 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 841/1681] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:04 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:05.180432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d6594-694e-4f91-8d04-f4a15321f0b6 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:05.180921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d6594-694e-4f91-8d04-f4a15321f0b6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:05.181028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9d6594-694e-4f91-8d04-f4a15321f0b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:05.253624 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-3f9d6594-694e-4f91-8d04-f4a15321f0b6 None None] MFA Rules not processed for user `d0427d9cce40423185a40df61c3617a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:05.264199 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3f9d6594-694e-4f91-8d04-f4a15321f0b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:05.417519 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385204c-de73-483b-9eb0-034c42d008e2 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:05.422477 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f385204c-de73-483b-9eb0-034c42d008e2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:05.617356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 841/1682] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:05 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 438 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:05.654422 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f385204c-de73-483b-9eb0-034c42d008e2 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:05.669883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67514fbb-f581-4e48-a46b-cc41a33dc6d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:05.670736 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67514fbb-f581-4e48-a46b-cc41a33dc6d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:05.671514 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67514fbb-f581-4e48-a46b-cc41a33dc6d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:05.671815 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67514fbb-f581-4e48-a46b-cc41a33dc6d3 admin admin] PATH_INFO: `/v3/users/84a88a406e7349478f31ee93b258eca0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:05.673739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67514fbb-f581-4e48-a46b-cc41a33dc6d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84a88a406e7349478f31ee93b258eca0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:05.674911 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: WARNING py.warnings [None req-67514fbb-f581-4e48-a46b-cc41a33dc6d3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required May 25 16:08:05.674911 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: warnings.warn(msg) May 25 16:08:05.674911 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: May 25 16:08:05.683929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67514fbb-f581-4e48-a46b-cc41a33dc6d3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:05.902166 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-67514fbb-f581-4e48-a46b-cc41a33dc6d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84a88a406e7349478f31ee93b258eca0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:05.989258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-67514fbb-f581-4e48-a46b-cc41a33dc6d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84a88a406e7349478f31ee93b258eca0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:06.014701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-67514fbb-f581-4e48-a46b-cc41a33dc6d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84a88a406e7349478f31ee93b258eca0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:06.015673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-67514fbb-f581-4e48-a46b-cc41a33dc6d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84a88a406e7349478f31ee93b258eca0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:06.069945 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-67514fbb-f581-4e48-a46b-cc41a33dc6d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84a88a406e7349478f31ee93b258eca0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:06.114900 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 842/1683] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:05 2022] DELETE /identity/v3/users/84a88a406e7349478f31ee93b258eca0 => generated 0 bytes in 698 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:06.119667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d97ec4-7239-4b28-b622-4022d57161df None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:06.120222 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d97ec4-7239-4b28-b622-4022d57161df None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:06.120694 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d97ec4-7239-4b28-b622-4022d57161df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:06.152316 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-33d97ec4-7239-4b28-b622-4022d57161df None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:06.158577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-33d97ec4-7239-4b28-b622-4022d57161df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:06.409879 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 842/1684] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 291 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:06.523873 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67514fbb-f581-4e48-a46b-cc41a33dc6d3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:06.525758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-67514fbb-f581-4e48-a46b-cc41a33dc6d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:06.639033 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:06.793434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67514fbb-f581-4e48-a46b-cc41a33dc6d3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:06.806682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6afb8-43ba-43cd-99dd-27a6a52892a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:06.807275 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6afb8-43ba-43cd-99dd-27a6a52892a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:06.807962 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6afb8-43ba-43cd-99dd-27a6a52892a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:06.810970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6afb8-43ba-43cd-99dd-27a6a52892a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:06.833617 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6afb8-43ba-43cd-99dd-27a6a52892a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:06.835583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6afb8-43ba-43cd-99dd-27a6a52892a2 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:06.869372 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 843/1685] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:07.190612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295fb8a-ed86-4517-aa76-6c5a768ec382 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:07.190612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295fb8a-ed86-4517-aa76-6c5a768ec382 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:07.190830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6295fb8a-ed86-4517-aa76-6c5a768ec382 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:07.263904 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6295fb8a-ed86-4517-aa76-6c5a768ec382 None None] MFA Rules not processed for user `708fdc537e1747d7bd5917ed7af9c368`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:07.265627 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6295fb8a-ed86-4517-aa76-6c5a768ec382 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:07.336271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:07.391978 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 843/1686] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:07.545750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6afb8-43ba-43cd-99dd-27a6a52892a2 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:07.549564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1de6afb8-43ba-43cd-99dd-27a6a52892a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:07.670358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6afb8-43ba-43cd-99dd-27a6a52892a2 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:07.679941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b92854-4564-4dbb-b692-3f579378d402 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:07.683843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b92854-4564-4dbb-b692-3f579378d402 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:07.684340 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b92854-4564-4dbb-b692-3f579378d402 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:07.686217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b92854-4564-4dbb-b692-3f579378d402 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:07.698284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b92854-4564-4dbb-b692-3f579378d402 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:07.700320 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b92854-4564-4dbb-b692-3f579378d402 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:07.731872 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 844/1687] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:08.345871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cad22-0c7d-4504-9b81-186992ff5522 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:08.346419 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cad22-0c7d-4504-9b81-186992ff5522 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:08.346524 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24cad22-0c7d-4504-9b81-186992ff5522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:08.381644 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d24cad22-0c7d-4504-9b81-186992ff5522 None None] MFA Rules not processed for user `be5c51fdd2fe4159a83a560dc074da6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:08.384747 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d24cad22-0c7d-4504-9b81-186992ff5522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:08.538990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 844/1688] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:08 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:08.631462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76098e-3511-4f79-9f20-b0619d3d99d0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:08.632097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76098e-3511-4f79-9f20-b0619d3d99d0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:08.632333 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db76098e-3511-4f79-9f20-b0619d3d99d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:08.735532 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-db76098e-3511-4f79-9f20-b0619d3d99d0 None None] MFA Rules not processed for user `a0289402c6f245839a86b7b66714a8f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:08.739055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-db76098e-3511-4f79-9f20-b0619d3d99d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:09.047931 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 845/1689] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:08 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:09.063572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cad22-0c7d-4504-9b81-186992ff5522 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:09.097536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24cad22-0c7d-4504-9b81-186992ff5522 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:09.122517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1c1bc-3181-4395-80b4-25ddf1511a4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:09.124099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1c1bc-3181-4395-80b4-25ddf1511a4a service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:09.127880 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1c1bc-3181-4395-80b4-25ddf1511a4a service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:09.132267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3e1c1bc-3181-4395-80b4-25ddf1511a4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:09.164529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76098e-3511-4f79-9f20-b0619d3d99d0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:09.166493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-db76098e-3511-4f79-9f20-b0619d3d99d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:09.167507 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1c1bc-3181-4395-80b4-25ddf1511a4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:09.174988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3e1c1bc-3181-4395-80b4-25ddf1511a4a service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:09.192143 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db76098e-3511-4f79-9f20-b0619d3d99d0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:09.206822 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6d8325-cf7d-43bb-b1dc-fbc28ab345aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:09.208118 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d8325-cf7d-43bb-b1dc-fbc28ab345aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:09.208636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d8325-cf7d-43bb-b1dc-fbc28ab345aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:09.209117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6d8325-cf7d-43bb-b1dc-fbc28ab345aa admin admin] PATH_INFO: `/v3/users/5d8147e581634e718814eb52886bcbb3` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:09.212889 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d8325-cf7d-43bb-b1dc-fbc28ab345aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d8147e581634e718814eb52886bcbb3)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:09.214030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 845/1690] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:09 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:09.217776 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6d8325-cf7d-43bb-b1dc-fbc28ab345aa admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:09.335506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:09.377515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bf6d8325-cf7d-43bb-b1dc-fbc28ab345aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d8147e581634e718814eb52886bcbb3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:09.431412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bf6d8325-cf7d-43bb-b1dc-fbc28ab345aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d8147e581634e718814eb52886bcbb3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:09.469660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1c1bc-3181-4395-80b4-25ddf1511a4a service nova] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:09.470146 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bf6d8325-cf7d-43bb-b1dc-fbc28ab345aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d8147e581634e718814eb52886bcbb3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:09.470948 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bf6d8325-cf7d-43bb-b1dc-fbc28ab345aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d8147e581634e718814eb52886bcbb3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:09.475852 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c3e1c1bc-3181-4395-80b4-25ddf1511a4a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:09.501400 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bf6d8325-cf7d-43bb-b1dc-fbc28ab345aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d8147e581634e718814eb52886bcbb3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:09.505343 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3e1c1bc-3181-4395-80b4-25ddf1511a4a service nova] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:09.517815 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de20e82f-fde7-486e-b68b-339930095e67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:09.520407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20e82f-fde7-486e-b68b-339930095e67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:09.522141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20e82f-fde7-486e-b68b-339930095e67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:09.523099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de20e82f-fde7-486e-b68b-339930095e67 admin admin] PATH_INFO: `/v3/projects/04d735ad2d88494f95cda019507da1df` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:09.535582 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de20e82f-fde7-486e-b68b-339930095e67 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04d735ad2d88494f95cda019507da1df)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:09.540356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de20e82f-fde7-486e-b68b-339930095e67 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:09.556562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 846/1691] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:09 2022] DELETE /identity/v3/users/5d8147e581634e718814eb52886bcbb3 => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:09.561588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3f845-a3d3-4106-bd02-c3540d6587e8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:09.564310 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3f845-a3d3-4106-bd02-c3540d6587e8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:09.572671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4a3f845-a3d3-4106-bd02-c3540d6587e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:09.688449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-de20e82f-fde7-486e-b68b-339930095e67 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04d735ad2d88494f95cda019507da1df', 'request_id': 'req-de20e82f-fde7-486e-b68b-339930095e67'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:09.706585 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-de20e82f-fde7-486e-b68b-339930095e67 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04d735ad2d88494f95cda019507da1df', 'request_id': 'req-de20e82f-fde7-486e-b68b-339930095e67'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:09.711873 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c4a3f845-a3d3-4106-bd02-c3540d6587e8 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:09.713146 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c4a3f845-a3d3-4106-bd02-c3540d6587e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:09.725141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-de20e82f-fde7-486e-b68b-339930095e67 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04d735ad2d88494f95cda019507da1df', 'request_id': 'req-de20e82f-fde7-486e-b68b-339930095e67'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:09.783150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-de20e82f-fde7-486e-b68b-339930095e67 admin admin] The token cache is being invalidate because project 04d735ad2d88494f95cda019507da1df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:09.783150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-de20e82f-fde7-486e-b68b-339930095e67 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:09.786293 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-de20e82f-fde7-486e-b68b-339930095e67 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:09.794501 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 846/1692] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:09 2022] DELETE /identity/v3/projects/04d735ad2d88494f95cda019507da1df => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:09.801262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1d785-98ee-408a-b8c2-686a2ff0d0c9 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:09.803230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1d785-98ee-408a-b8c2-686a2ff0d0c9 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:09.803607 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d1d785-98ee-408a-b8c2-686a2ff0d0c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:09.896617 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-86d1d785-98ee-408a-b8c2-686a2ff0d0c9 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:09.900908 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-86d1d785-98ee-408a-b8c2-686a2ff0d0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:09.974435 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 847/1693] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 415 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:09.977836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8cdc6-1613-4bad-86d7-555072df0c29 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:09.978928 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8cdc6-1613-4bad-86d7-555072df0c29 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:09.979438 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce8cdc6-1613-4bad-86d7-555072df0c29 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:10.053741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3ce8cdc6-1613-4bad-86d7-555072df0c29 None None] MFA Rules not processed for user `413b3d65427b41dc99bc7b0a94fb1dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:10.061133 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:10.065170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3ce8cdc6-1613-4bad-86d7-555072df0c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:10.106982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 847/1694] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:10.112205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1d785-98ee-408a-b8c2-686a2ff0d0c9 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:10.119225 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-86d1d785-98ee-408a-b8c2-686a2ff0d0c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:10.305505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d1d785-98ee-408a-b8c2-686a2ff0d0c9 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:10.322666 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b63dfc3-367e-40fa-80e2-27c1a3ff0e71 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:10.323291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63dfc3-367e-40fa-80e2-27c1a3ff0e71 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:10.323584 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63dfc3-367e-40fa-80e2-27c1a3ff0e71 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:10.325570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b63dfc3-367e-40fa-80e2-27c1a3ff0e71 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:10.350020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b63dfc3-367e-40fa-80e2-27c1a3ff0e71 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:10.355231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b63dfc3-367e-40fa-80e2-27c1a3ff0e71 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:10.426183 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 848/1695] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:10.432573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9133708-6360-4c97-ab68-fc4c530765ae None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:10.433340 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9133708-6360-4c97-ab68-fc4c530765ae None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:10.433477 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9133708-6360-4c97-ab68-fc4c530765ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:10.501383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b9133708-6360-4c97-ab68-fc4c530765ae None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:10.503739 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b9133708-6360-4c97-ab68-fc4c530765ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:10.533120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 849/1696] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:10.534813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9133708-6360-4c97-ab68-fc4c530765ae None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:10.565454 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9133708-6360-4c97-ab68-fc4c530765ae None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:10.580500 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b4f03-c04f-44a8-a34c-b7d49c1be830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:10.581657 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b4f03-c04f-44a8-a34c-b7d49c1be830 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:10.582572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b4f03-c04f-44a8-a34c-b7d49c1be830 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:10.583083 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a2b4f03-c04f-44a8-a34c-b7d49c1be830 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:10.585351 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b4f03-c04f-44a8-a34c-b7d49c1be830 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:10.587085 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b4f03-c04f-44a8-a34c-b7d49c1be830 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:10.589172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a2b4f03-c04f-44a8-a34c-b7d49c1be830 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:10.637715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 850/1697] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:10.639348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b4f03-c04f-44a8-a34c-b7d49c1be830 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:10.663263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a2b4f03-c04f-44a8-a34c-b7d49c1be830 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:10.670138 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0958c-904f-4f22-8168-fde8111f3bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:10.671337 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0958c-904f-4f22-8168-fde8111f3bb1 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:10.671961 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0958c-904f-4f22-8168-fde8111f3bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:10.672569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1af0958c-904f-4f22-8168-fde8111f3bb1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:10.673654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af0958c-904f-4f22-8168-fde8111f3bb1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:10.676365 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1af0958c-904f-4f22-8168-fde8111f3bb1 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:10.795560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 851/1698] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:10 2022] POST /identity/v3/projects => generated 367 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:10.797859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0958c-904f-4f22-8168-fde8111f3bb1 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:10.828037 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 848/1699] 10.222.0.67 () {58 vars in 1015 bytes} [Wed May 25 16:08:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 852 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:10.941487 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8cdc6-1613-4bad-86d7-555072df0c29 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:10.943046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3ce8cdc6-1613-4bad-86d7-555072df0c29 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:10.969760 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce8cdc6-1613-4bad-86d7-555072df0c29 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:10.995808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1af0958c-904f-4f22-8168-fde8111f3bb1 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:11.011582 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa79450-0c9c-4053-95af-77be95af29b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:11.012419 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa79450-0c9c-4053-95af-77be95af29b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:11.012763 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa79450-0c9c-4053-95af-77be95af29b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:11.013110 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa79450-0c9c-4053-95af-77be95af29b0 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:11.013901 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa79450-0c9c-4053-95af-77be95af29b0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:11.015999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa79450-0c9c-4053-95af-77be95af29b0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:11.066270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 852/1700] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:10 2022] POST /identity/v3/users => generated 284 bytes in 269 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:11.073142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa79450-0c9c-4053-95af-77be95af29b0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:11.090861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa79450-0c9c-4053-95af-77be95af29b0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:11.098150 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681cb4c-0553-41c4-8c10-4a5c77672b85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:11.100932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681cb4c-0553-41c4-8c10-4a5c77672b85 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:11.101095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681cb4c-0553-41c4-8c10-4a5c77672b85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:11.101514 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5681cb4c-0553-41c4-8c10-4a5c77672b85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:11.106272 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0d7bd-01a6-4d73-85f8-a1e326454b44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:11.107087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0d7bd-01a6-4d73-85f8-a1e326454b44 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:11.107544 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0d7bd-01a6-4d73-85f8-a1e326454b44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:11.108588 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c0d7bd-01a6-4d73-85f8-a1e326454b44 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:11.108708 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0d7bd-01a6-4d73-85f8-a1e326454b44 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:11.113571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c0d7bd-01a6-4d73-85f8-a1e326454b44 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:11.125237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681cb4c-0553-41c4-8c10-4a5c77672b85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:11.127262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5681cb4c-0553-41c4-8c10-4a5c77672b85 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:11.142332 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 853/1701] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:11.143336 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0d7bd-01a6-4d73-85f8-a1e326454b44 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:11.151675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 849/1702] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:10 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:08:11.178302 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c0d7bd-01a6-4d73-85f8-a1e326454b44 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:11.187932 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8626f4-f1c7-4bc7-96a1-b060591c0955 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:11.188287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8626f4-f1c7-4bc7-96a1-b060591c0955 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:11.188524 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8626f4-f1c7-4bc7-96a1-b060591c0955 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:11.188781 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8626f4-f1c7-4bc7-96a1-b060591c0955 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:11.189532 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8626f4-f1c7-4bc7-96a1-b060591c0955 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:11.191223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8626f4-f1c7-4bc7-96a1-b060591c0955 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:11.218079 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 854/1703] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:11 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:11.222714 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681cb4c-0553-41c4-8c10-4a5c77672b85 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:11.234668 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:11.241873 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5681cb4c-0553-41c4-8c10-4a5c77672b85 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:11.258889 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ea369-76ce-4ba9-96e4-4200f958a84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:11.260273 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ea369-76ce-4ba9-96e4-4200f958a84d admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:11.260669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ea369-76ce-4ba9-96e4-4200f958a84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:11.261056 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b19ea369-76ce-4ba9-96e4-4200f958a84d admin admin] PATH_INFO: `/v3/projects/30f8d2e80d6342ca90f689fbb3b78d5b/users/dfa1bae9ee9f47e1b330391b90bc4fd1/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:11.311640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ea369-76ce-4ba9-96e4-4200f958a84d admin admin] RBAC: Authorizing `identity:create_grant(project_id=30f8d2e80d6342ca90f689fbb3b78d5b, user_id=dfa1bae9ee9f47e1b330391b90bc4fd1, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:11.314877 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b19ea369-76ce-4ba9-96e4-4200f958a84d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:11.345277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 850/1704] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:11 2022] PUT /identity/v3/projects/30f8d2e80d6342ca90f689fbb3b78d5b/users/dfa1bae9ee9f47e1b330391b90bc4fd1/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:11.357354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c710-f93f-40e0-adbe-5695e0f1237d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:11.357765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c710-f93f-40e0-adbe-5695e0f1237d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:11.358112 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e021c710-f93f-40e0-adbe-5695e0f1237d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:11.404902 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ea369-76ce-4ba9-96e4-4200f958a84d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:11.407151 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b19ea369-76ce-4ba9-96e4-4200f958a84d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:11.430986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e021c710-f93f-40e0-adbe-5695e0f1237d None None] MFA Rules not processed for user `dfa1bae9ee9f47e1b330391b90bc4fd1`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:11.434445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e021c710-f93f-40e0-adbe-5695e0f1237d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:11.489529 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:11.571098 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b19ea369-76ce-4ba9-96e4-4200f958a84d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:11.583030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d971d4e-9346-4363-badf-ee4f4ece73ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:11.585924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d971d4e-9346-4363-badf-ee4f4ece73ad admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:11.586793 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d971d4e-9346-4363-badf-ee4f4ece73ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:11.587231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d971d4e-9346-4363-badf-ee4f4ece73ad admin admin] PATH_INFO: `/v3/projects/072316d170de4f50a27886d804fa252a` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:11.590868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d971d4e-9346-4363-badf-ee4f4ece73ad admin admin] RBAC: Authorizing `identity:delete_project(project_id=072316d170de4f50a27886d804fa252a)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:11.594760 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d971d4e-9346-4363-badf-ee4f4ece73ad admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:11.608763 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 855/1705] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:11 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:11.611620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021c710-f93f-40e0-adbe-5695e0f1237d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:11.635259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e021c710-f93f-40e0-adbe-5695e0f1237d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:11.646826 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89f92f-21c2-4660-9c19-6002758d806c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:11.649172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89f92f-21c2-4660-9c19-6002758d806c admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:11.650443 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89f92f-21c2-4660-9c19-6002758d806c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:11.651195 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c89f92f-21c2-4660-9c19-6002758d806c admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:11.653743 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89f92f-21c2-4660-9c19-6002758d806c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:11.657412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c89f92f-21c2-4660-9c19-6002758d806c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:11.721883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3d971d4e-9346-4363-badf-ee4f4ece73ad admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '072316d170de4f50a27886d804fa252a', 'request_id': 'req-3d971d4e-9346-4363-badf-ee4f4ece73ad'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:11.742488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 856/1706] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:11 2022] POST /identity/v3/projects => generated 369 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:11.744101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3d971d4e-9346-4363-badf-ee4f4ece73ad admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '072316d170de4f50a27886d804fa252a', 'request_id': 'req-3d971d4e-9346-4363-badf-ee4f4ece73ad'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:11.746704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89f92f-21c2-4660-9c19-6002758d806c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:11.762498 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3d971d4e-9346-4363-badf-ee4f4ece73ad admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '072316d170de4f50a27886d804fa252a', 'request_id': 'req-3d971d4e-9346-4363-badf-ee4f4ece73ad'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:11.806024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3d971d4e-9346-4363-badf-ee4f4ece73ad admin admin] The token cache is being invalidate because project 072316d170de4f50a27886d804fa252a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:11.808987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3d971d4e-9346-4363-badf-ee4f4ece73ad admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:11.813693 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3d971d4e-9346-4363-badf-ee4f4ece73ad admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:11.835220 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 851/1707] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:11 2022] DELETE /identity/v3/projects/072316d170de4f50a27886d804fa252a => generated 0 bytes in 428 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:11.838610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d971d4e-9346-4363-badf-ee4f4ece73ad admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:11.845573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3d971d4e-9346-4363-badf-ee4f4ece73ad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:12.021413 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c89f92f-21c2-4660-9c19-6002758d806c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:12.034759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9dda5-e7a7-4eca-a42a-e1594ba0f072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:12.035482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9dda5-e7a7-4eca-a42a-e1594ba0f072 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:12.037373 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9dda5-e7a7-4eca-a42a-e1594ba0f072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:12.037667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c9dda5-e7a7-4eca-a42a-e1594ba0f072 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:12.041112 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9dda5-e7a7-4eca-a42a-e1594ba0f072 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:12.047139 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41c9dda5-e7a7-4eca-a42a-e1594ba0f072 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:12.082487 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d971d4e-9346-4363-badf-ee4f4ece73ad admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:12.104738 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 857/1708] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:11 2022] POST /identity/v3/users => generated 291 bytes in 358 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:12.106637 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9dda5-e7a7-4eca-a42a-e1594ba0f072 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:12.111040 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-41c9dda5-e7a7-4eca-a42a-e1594ba0f072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:12.113061 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2d4de-21ca-4e22-8c37-fff2b8b081ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:12.114125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2d4de-21ca-4e22-8c37-fff2b8b081ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:12.114410 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2d4de-21ca-4e22-8c37-fff2b8b081ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:12.114688 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a2d4de-21ca-4e22-8c37-fff2b8b081ed admin admin] PATH_INFO: `/v3/users/06cbac33fe154aff9c1e2e8baf1b37a9` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:12.118355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2d4de-21ca-4e22-8c37-fff2b8b081ed admin admin] RBAC: Authorizing `identity:delete_user(user_id=06cbac33fe154aff9c1e2e8baf1b37a9)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:12.123157 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a2d4de-21ca-4e22-8c37-fff2b8b081ed admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:12.145306 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c9dda5-e7a7-4eca-a42a-e1594ba0f072 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:12.158733 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d82d56-e8ea-434a-a3fe-914893ce2879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:12.159646 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d82d56-e8ea-434a-a3fe-914893ce2879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:12.160585 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:12.165129 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d82d56-e8ea-434a-a3fe-914893ce2879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:12.165535 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40d82d56-e8ea-434a-a3fe-914893ce2879 admin admin] PATH_INFO: `/v3/projects/684290bd47864be38ec9736193b7e1ce` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:12.170060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d82d56-e8ea-434a-a3fe-914893ce2879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=684290bd47864be38ec9736193b7e1ce)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:12.180307 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40d82d56-e8ea-434a-a3fe-914893ce2879 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:12.308399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-17a2d4de-21ca-4e22-8c37-fff2b8b081ed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06cbac33fe154aff9c1e2e8baf1b37a9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:12.369695 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-17a2d4de-21ca-4e22-8c37-fff2b8b081ed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06cbac33fe154aff9c1e2e8baf1b37a9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:12.395921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-17a2d4de-21ca-4e22-8c37-fff2b8b081ed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06cbac33fe154aff9c1e2e8baf1b37a9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:12.396631 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-17a2d4de-21ca-4e22-8c37-fff2b8b081ed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06cbac33fe154aff9c1e2e8baf1b37a9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:12.432228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-40d82d56-e8ea-434a-a3fe-914893ce2879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '684290bd47864be38ec9736193b7e1ce', 'request_id': 'req-40d82d56-e8ea-434a-a3fe-914893ce2879'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:12.449505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-17a2d4de-21ca-4e22-8c37-fff2b8b081ed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06cbac33fe154aff9c1e2e8baf1b37a9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:12.460581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-40d82d56-e8ea-434a-a3fe-914893ce2879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '684290bd47864be38ec9736193b7e1ce', 'request_id': 'req-40d82d56-e8ea-434a-a3fe-914893ce2879'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:12.475141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 852/1709] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:11 2022] DELETE /identity/v3/users/06cbac33fe154aff9c1e2e8baf1b37a9 => generated 0 bytes in 636 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:12.475616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2d4de-21ca-4e22-8c37-fff2b8b081ed admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:12.480386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-17a2d4de-21ca-4e22-8c37-fff2b8b081ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:12.481002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-40d82d56-e8ea-434a-a3fe-914893ce2879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '684290bd47864be38ec9736193b7e1ce', 'request_id': 'req-40d82d56-e8ea-434a-a3fe-914893ce2879'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:12.539367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-40d82d56-e8ea-434a-a3fe-914893ce2879 admin admin] The token cache is being invalidate because project 684290bd47864be38ec9736193b7e1ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:12.543288 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-40d82d56-e8ea-434a-a3fe-914893ce2879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:12.544815 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-40d82d56-e8ea-434a-a3fe-914893ce2879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:12.547111 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 858/1710] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:12 2022] DELETE /identity/v3/projects/684290bd47864be38ec9736193b7e1ce => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:12.551241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d82d56-e8ea-434a-a3fe-914893ce2879 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:12.563205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-40d82d56-e8ea-434a-a3fe-914893ce2879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:12.592004 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=88,keystone:INSERT=5,keystone:DELETE=7 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:12.744566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a2d4de-21ca-4e22-8c37-fff2b8b081ed admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:12.750446 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e0677-3f2c-46be-91d6-58572d2059aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:12.751114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e0677-3f2c-46be-91d6-58572d2059aa admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:12.751520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e0677-3f2c-46be-91d6-58572d2059aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:12.751705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235e0677-3f2c-46be-91d6-58572d2059aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:12.756301 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e0677-3f2c-46be-91d6-58572d2059aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:12.761889 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235e0677-3f2c-46be-91d6-58572d2059aa admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:12.787271 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 853/1711] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:12.791482 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e0677-3f2c-46be-91d6-58572d2059aa admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:12.795155 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-235e0677-3f2c-46be-91d6-58572d2059aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:12.806142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40d82d56-e8ea-434a-a3fe-914893ce2879 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:12.826184 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235e0677-3f2c-46be-91d6-58572d2059aa admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:12.830183 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc295f-775f-43a7-9c8f-24f8cfdcf94d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:12.831257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc295f-775f-43a7-9c8f-24f8cfdcf94d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:12.831596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc295f-775f-43a7-9c8f-24f8cfdcf94d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:12.833878 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bcc295f-775f-43a7-9c8f-24f8cfdcf94d admin admin] PATH_INFO: `/v3/users/7765d0e3cb5e404199840c867260c1b1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:12.837882 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc295f-775f-43a7-9c8f-24f8cfdcf94d admin admin] RBAC: Authorizing `identity:delete_user(user_id=7765d0e3cb5e404199840c867260c1b1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:12.839719 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789aa0e-813c-4bee-b55e-f6d2f684a55a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:12.842451 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789aa0e-813c-4bee-b55e-f6d2f684a55a admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:12.843234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789aa0e-813c-4bee-b55e-f6d2f684a55a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:12.843567 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bcc295f-775f-43a7-9c8f-24f8cfdcf94d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:12.844661 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4789aa0e-813c-4bee-b55e-f6d2f684a55a admin admin] PATH_INFO: `/v3/projects/421601a3bf9b43c6914d2fea9e75e62f/users/3c938ac58c1d4cb09e76b9f6fd61c0d8/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:12.897497 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789aa0e-813c-4bee-b55e-f6d2f684a55a admin admin] RBAC: Authorizing `identity:create_grant(project_id=421601a3bf9b43c6914d2fea9e75e62f, user_id=3c938ac58c1d4cb09e76b9f6fd61c0d8, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:12.905881 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4789aa0e-813c-4bee-b55e-f6d2f684a55a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:12.937218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 854/1712] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:12 2022] PUT /identity/v3/projects/421601a3bf9b43c6914d2fea9e75e62f/users/3c938ac58c1d4cb09e76b9f6fd61c0d8/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:12.940989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789aa0e-813c-4bee-b55e-f6d2f684a55a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:13.033337 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8bcc295f-775f-43a7-9c8f-24f8cfdcf94d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7765d0e3cb5e404199840c867260c1b1'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:13.034142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8bcc295f-775f-43a7-9c8f-24f8cfdcf94d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7765d0e3cb5e404199840c867260c1b1'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:13.062047 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8bcc295f-775f-43a7-9c8f-24f8cfdcf94d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7765d0e3cb5e404199840c867260c1b1'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:13.080246 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8bcc295f-775f-43a7-9c8f-24f8cfdcf94d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7765d0e3cb5e404199840c867260c1b1'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:13.081035 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8bcc295f-775f-43a7-9c8f-24f8cfdcf94d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7765d0e3cb5e404199840c867260c1b1'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:13.110314 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4789aa0e-813c-4bee-b55e-f6d2f684a55a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:13.117964 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 859/1713] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:12 2022] DELETE /identity/v3/users/7765d0e3cb5e404199840c867260c1b1 => generated 0 bytes in 564 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:13.131736 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ba8470-13a1-4a72-a69e-95f8b4bc167c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:13.131736 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba8470-13a1-4a72-a69e-95f8b4bc167c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:13.131736 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba8470-13a1-4a72-a69e-95f8b4bc167c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:13.131736 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ba8470-13a1-4a72-a69e-95f8b4bc167c admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:13.139088 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba8470-13a1-4a72-a69e-95f8b4bc167c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:13.144073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ba8470-13a1-4a72-a69e-95f8b4bc167c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:13.154576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:13.178412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc295f-775f-43a7-9c8f-24f8cfdcf94d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:13.178753 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 855/1714] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:12 2022] GET /identity/v3/roles => generated 1420 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:13.180526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8bcc295f-775f-43a7-9c8f-24f8cfdcf94d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:13.330102 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bcc295f-775f-43a7-9c8f-24f8cfdcf94d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:13.335985 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe53e48-e5ec-4132-87eb-e69e4ebfbdbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:13.338775 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe53e48-e5ec-4132-87eb-e69e4ebfbdbd admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:13.339087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe53e48-e5ec-4132-87eb-e69e4ebfbdbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:13.339346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe53e48-e5ec-4132-87eb-e69e4ebfbdbd admin admin] PATH_INFO: `/v3/projects/421601a3bf9b43c6914d2fea9e75e62f/users/3c938ac58c1d4cb09e76b9f6fd61c0d8/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:13.349423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe53e48-e5ec-4132-87eb-e69e4ebfbdbd admin admin] RBAC: Authorizing `identity:create_grant(project_id=421601a3bf9b43c6914d2fea9e75e62f, user_id=3c938ac58c1d4cb09e76b9f6fd61c0d8, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:13.352413 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fe53e48-e5ec-4132-87eb-e69e4ebfbdbd admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:13.382516 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 860/1715] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:13 2022] PUT /identity/v3/projects/421601a3bf9b43c6914d2fea9e75e62f/users/3c938ac58c1d4cb09e76b9f6fd61c0d8/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:13.397139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3d10f-3324-4471-bb09-217160961da6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:13.402990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3d10f-3324-4471-bb09-217160961da6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:13.403848 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb3d10f-3324-4471-bb09-217160961da6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:13.491846 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-adb3d10f-3324-4471-bb09-217160961da6 None None] MFA Rules not processed for user `3c938ac58c1d4cb09e76b9f6fd61c0d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:13.493939 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-adb3d10f-3324-4471-bb09-217160961da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:13.704187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 856/1716] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:13 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 310 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:13.712136 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe53e48-e5ec-4132-87eb-e69e4ebfbdbd admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:13.713951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2fe53e48-e5ec-4132-87eb-e69e4ebfbdbd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:13.760285 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:13.851108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fe53e48-e5ec-4132-87eb-e69e4ebfbdbd admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:13.860932 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566b911-fae0-4511-a25d-fd2bbe8f9469 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:13.861640 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566b911-fae0-4511-a25d-fd2bbe8f9469 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:13.861932 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566b911-fae0-4511-a25d-fd2bbe8f9469 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:13.862209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c566b911-fae0-4511-a25d-fd2bbe8f9469 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:13.900380 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566b911-fae0-4511-a25d-fd2bbe8f9469 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:13.901924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c566b911-fae0-4511-a25d-fd2bbe8f9469 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:13.947383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 861/1717] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:13 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 236 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:14.020605 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3d10f-3324-4471-bb09-217160961da6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:14.022849 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-adb3d10f-3324-4471-bb09-217160961da6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:14.073469 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb3d10f-3324-4471-bb09-217160961da6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:14.094381 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7234e3db-bd4a-416f-b05e-d74cc062f1f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:14.095158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234e3db-bd4a-416f-b05e-d74cc062f1f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:14.095328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234e3db-bd4a-416f-b05e-d74cc062f1f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:14.095610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7234e3db-bd4a-416f-b05e-d74cc062f1f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:14.103753 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7234e3db-bd4a-416f-b05e-d74cc062f1f6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:14.319386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234e3db-bd4a-416f-b05e-d74cc062f1f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:14.321328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7234e3db-bd4a-416f-b05e-d74cc062f1f6 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:14.347527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 857/1718] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:14 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:15.153253 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566b911-fae0-4511-a25d-fd2bbe8f9469 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:15.154727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c566b911-fae0-4511-a25d-fd2bbe8f9469 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:15.391077 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c566b911-fae0-4511-a25d-fd2bbe8f9469 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:15.421508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2f105-232d-4114-a0d9-f78cc1761e58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:15.423557 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2f105-232d-4114-a0d9-f78cc1761e58 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:15.424443 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2f105-232d-4114-a0d9-f78cc1761e58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:15.424855 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f2f105-232d-4114-a0d9-f78cc1761e58 admin admin] PATH_INFO: `/v3/projects/5500e045282246458b03e691f0201623` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:15.426889 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2f105-232d-4114-a0d9-f78cc1761e58 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5500e045282246458b03e691f0201623)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:15.429050 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f2f105-232d-4114-a0d9-f78cc1761e58 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:15.558911 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-43f2f105-232d-4114-a0d9-f78cc1761e58 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5500e045282246458b03e691f0201623', 'request_id': 'req-43f2f105-232d-4114-a0d9-f78cc1761e58'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:15.586350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-43f2f105-232d-4114-a0d9-f78cc1761e58 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5500e045282246458b03e691f0201623', 'request_id': 'req-43f2f105-232d-4114-a0d9-f78cc1761e58'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:15.607288 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-43f2f105-232d-4114-a0d9-f78cc1761e58 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5500e045282246458b03e691f0201623', 'request_id': 'req-43f2f105-232d-4114-a0d9-f78cc1761e58'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:15.637514 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-43f2f105-232d-4114-a0d9-f78cc1761e58 admin admin] The token cache is being invalidate because project 5500e045282246458b03e691f0201623 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:15.638111 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-43f2f105-232d-4114-a0d9-f78cc1761e58 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:15.639518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-43f2f105-232d-4114-a0d9-f78cc1761e58 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:15.642054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 862/1719] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:15 2022] DELETE /identity/v3/projects/5500e045282246458b03e691f0201623 => generated 0 bytes in 489 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:15.647868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cc45e-4d72-4728-a821-3c561c94394e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:15.648413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cc45e-4d72-4728-a821-3c561c94394e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:15.648575 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52cc45e-4d72-4728-a821-3c561c94394e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:15.689323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e52cc45e-4d72-4728-a821-3c561c94394e None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:15.693111 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e52cc45e-4d72-4728-a821-3c561c94394e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:15.706342 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2f105-232d-4114-a0d9-f78cc1761e58 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:15.708258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-43f2f105-232d-4114-a0d9-f78cc1761e58 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:15.823935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 858/1720] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:15.833939 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f9333-4bc5-49a0-9859-a4b13de0d690 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:15.834365 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f9333-4bc5-49a0-9859-a4b13de0d690 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:15.834715 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-986f9333-4bc5-49a0-9859-a4b13de0d690 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:15.886935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-986f9333-4bc5-49a0-9859-a4b13de0d690 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:15.888783 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-986f9333-4bc5-49a0-9859-a4b13de0d690 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:15.899955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:15.921363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 859/1721] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:15.926182 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f9333-4bc5-49a0-9859-a4b13de0d690 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:15.951539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f2f105-232d-4114-a0d9-f78cc1761e58 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:15.955607 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-986f9333-4bc5-49a0-9859-a4b13de0d690 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:15.965611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edbd8f2b-a862-4248-8641-6b0405a5dc99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:15.967597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd8f2b-a862-4248-8641-6b0405a5dc99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:15.968062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd8f2b-a862-4248-8641-6b0405a5dc99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:15.968825 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edbd8f2b-a862-4248-8641-6b0405a5dc99 admin admin] PATH_INFO: `/v3/projects/3bd7991d000c47b794ff72c5be832231` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:15.969650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662440f1-2da2-4a8f-8889-ae27fd796f04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:15.970549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662440f1-2da2-4a8f-8889-ae27fd796f04 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:15.971036 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662440f1-2da2-4a8f-8889-ae27fd796f04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:15.971517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662440f1-2da2-4a8f-8889-ae27fd796f04 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:15.971999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd8f2b-a862-4248-8641-6b0405a5dc99 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3bd7991d000c47b794ff72c5be832231)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:15.974131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662440f1-2da2-4a8f-8889-ae27fd796f04 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:15.975401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662440f1-2da2-4a8f-8889-ae27fd796f04 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:15.980170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662440f1-2da2-4a8f-8889-ae27fd796f04 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:15.980554 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edbd8f2b-a862-4248-8641-6b0405a5dc99 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:16.019771 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 860/1722] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:08:16.025201 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662440f1-2da2-4a8f-8889-ae27fd796f04 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:16.037269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:16.049521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662440f1-2da2-4a8f-8889-ae27fd796f04 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:16.065227 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d071cc5-5183-4ab0-b92b-1bb8b90f0485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:16.067474 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d071cc5-5183-4ab0-b92b-1bb8b90f0485 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:16.068672 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d071cc5-5183-4ab0-b92b-1bb8b90f0485 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:16.069714 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d071cc5-5183-4ab0-b92b-1bb8b90f0485 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:16.073622 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d071cc5-5183-4ab0-b92b-1bb8b90f0485 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:16.078276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d071cc5-5183-4ab0-b92b-1bb8b90f0485 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:16.134438 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-edbd8f2b-a862-4248-8641-6b0405a5dc99 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3bd7991d000c47b794ff72c5be832231', 'request_id': 'req-edbd8f2b-a862-4248-8641-6b0405a5dc99'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:16.148285 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-edbd8f2b-a862-4248-8641-6b0405a5dc99 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3bd7991d000c47b794ff72c5be832231', 'request_id': 'req-edbd8f2b-a862-4248-8641-6b0405a5dc99'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:16.166485 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 861/1723] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:16 2022] POST /identity/v3/projects => generated 379 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:16.171025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-edbd8f2b-a862-4248-8641-6b0405a5dc99 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3bd7991d000c47b794ff72c5be832231', 'request_id': 'req-edbd8f2b-a862-4248-8641-6b0405a5dc99'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:16.174982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d071cc5-5183-4ab0-b92b-1bb8b90f0485 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:16.207944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-edbd8f2b-a862-4248-8641-6b0405a5dc99 admin admin] The token cache is being invalidate because project 3bd7991d000c47b794ff72c5be832231 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:16.208571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-edbd8f2b-a862-4248-8641-6b0405a5dc99 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:16.212115 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-edbd8f2b-a862-4248-8641-6b0405a5dc99 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:16.218426 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 863/1724] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:15 2022] DELETE /identity/v3/projects/3bd7991d000c47b794ff72c5be832231 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:16.223315 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ab001-51ac-4fa6-bb94-242d8fb3da9c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:16.225333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ab001-51ac-4fa6-bb94-242d8fb3da9c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:16.225740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261ab001-51ac-4fa6-bb94-242d8fb3da9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:16.307368 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-261ab001-51ac-4fa6-bb94-242d8fb3da9c None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:16.308744 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-261ab001-51ac-4fa6-bb94-242d8fb3da9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:16.330312 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d071cc5-5183-4ab0-b92b-1bb8b90f0485 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:16.340293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32305b5a-32ae-47f1-a7e3-e1b176b99e88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:16.341100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32305b5a-32ae-47f1-a7e3-e1b176b99e88 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:16.341426 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32305b5a-32ae-47f1-a7e3-e1b176b99e88 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:16.341710 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32305b5a-32ae-47f1-a7e3-e1b176b99e88 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:16.342475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32305b5a-32ae-47f1-a7e3-e1b176b99e88 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:16.352013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32305b5a-32ae-47f1-a7e3-e1b176b99e88 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:16.352347 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 864/1725] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:16.356446 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d31964-3d7a-4a04-b6b0-b9f225eab547 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:16.356873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d31964-3d7a-4a04-b6b0-b9f225eab547 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:16.357255 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52d31964-3d7a-4a04-b6b0-b9f225eab547 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:16.378459 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 862/1726] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:16 2022] POST /identity/v3/users => generated 290 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:16.381730 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32305b5a-32ae-47f1-a7e3-e1b176b99e88 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:16.386363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-32305b5a-32ae-47f1-a7e3-e1b176b99e88 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:16.414140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-52d31964-3d7a-4a04-b6b0-b9f225eab547 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:16.420737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-52d31964-3d7a-4a04-b6b0-b9f225eab547 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:16.424445 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32305b5a-32ae-47f1-a7e3-e1b176b99e88 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:16.438944 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09627f69-a816-4079-b0a3-a9df9db18ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:16.442057 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09627f69-a816-4079-b0a3-a9df9db18ad0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:16.447550 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09627f69-a816-4079-b0a3-a9df9db18ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:16.448403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 865/1727] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:16.449021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09627f69-a816-4079-b0a3-a9df9db18ad0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:16.451025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d31964-3d7a-4a04-b6b0-b9f225eab547 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:16.452167 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09627f69-a816-4079-b0a3-a9df9db18ad0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:16.455360 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09627f69-a816-4079-b0a3-a9df9db18ad0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:16.473969 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52d31964-3d7a-4a04-b6b0-b9f225eab547 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:16.486867 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 863/1728] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:16.488361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09627f69-a816-4079-b0a3-a9df9db18ad0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:16.489754 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5cc82-e2b5-43af-82fe-960e76b8b458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:16.492380 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5cc82-e2b5-43af-82fe-960e76b8b458 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:16.493055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5cc82-e2b5-43af-82fe-960e76b8b458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:16.495008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a5cc82-e2b5-43af-82fe-960e76b8b458 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:16.496681 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5cc82-e2b5-43af-82fe-960e76b8b458 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:16.498611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5cc82-e2b5-43af-82fe-960e76b8b458 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:16.504375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a5cc82-e2b5-43af-82fe-960e76b8b458 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:16.531228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09627f69-a816-4079-b0a3-a9df9db18ad0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:16.552776 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199191d-a812-47f9-8e41-00bf2faa65de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:16.555347 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199191d-a812-47f9-8e41-00bf2faa65de admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:16.558781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199191d-a812-47f9-8e41-00bf2faa65de admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:16.559412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8199191d-a812-47f9-8e41-00bf2faa65de admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:16.564268 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199191d-a812-47f9-8e41-00bf2faa65de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:16.566937 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 866/1729] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:16.569709 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8199191d-a812-47f9-8e41-00bf2faa65de admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:16.572277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5cc82-e2b5-43af-82fe-960e76b8b458 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:16.599321 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a5cc82-e2b5-43af-82fe-960e76b8b458 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:16.605624 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 864/1730] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:16.609193 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199191d-a812-47f9-8e41-00bf2faa65de admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:16.614240 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb45e8e-2063-48ec-ad78-a49ff046a38e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:16.617610 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb45e8e-2063-48ec-ad78-a49ff046a38e admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:16.619907 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb45e8e-2063-48ec-ad78-a49ff046a38e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:16.621059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eb45e8e-2063-48ec-ad78-a49ff046a38e admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:16.623150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb45e8e-2063-48ec-ad78-a49ff046a38e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:16.628428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eb45e8e-2063-48ec-ad78-a49ff046a38e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:16.640496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8199191d-a812-47f9-8e41-00bf2faa65de admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:16.653874 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6954f023-08dc-474f-813e-1ffef8dbe543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:16.656000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954f023-08dc-474f-813e-1ffef8dbe543 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:16.656000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954f023-08dc-474f-813e-1ffef8dbe543 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:16.664332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6954f023-08dc-474f-813e-1ffef8dbe543 admin admin] PATH_INFO: `/v3/projects/762c34908bf744d385b516f5b8d87b0f/users/7605790928584006a2da23db8afb2b95/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:16.698592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 867/1731] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:16 2022] POST /identity/v3/projects => generated 369 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:16.699996 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954f023-08dc-474f-813e-1ffef8dbe543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=762c34908bf744d385b516f5b8d87b0f, user_id=7605790928584006a2da23db8afb2b95, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:16.702727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb45e8e-2063-48ec-ad78-a49ff046a38e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:16.710169 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6954f023-08dc-474f-813e-1ffef8dbe543 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:16.745113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 865/1732] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:16 2022] PUT /identity/v3/projects/762c34908bf744d385b516f5b8d87b0f/users/7605790928584006a2da23db8afb2b95/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:16.760730 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae11bc-16e3-414c-837e-c570abcd1b13 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:16.761034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae11bc-16e3-414c-837e-c570abcd1b13 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:16.762542 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ae11bc-16e3-414c-837e-c570abcd1b13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:16.806823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c5ae11bc-16e3-414c-837e-c570abcd1b13 None None] MFA Rules not processed for user `7605790928584006a2da23db8afb2b95`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:16.821929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c5ae11bc-16e3-414c-837e-c570abcd1b13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:16.939325 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eb45e8e-2063-48ec-ad78-a49ff046a38e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:17.108784 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 866/1733] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:17.113848 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae11bc-16e3-414c-837e-c570abcd1b13 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:17.139785 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:17.164118 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:17.203850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9b99c-f609-4d1e-8111-dec58dc61c55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:17.204788 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9b99c-f609-4d1e-8111-dec58dc61c55 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:17.205185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9b99c-f609-4d1e-8111-dec58dc61c55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:17.205717 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c9b99c-f609-4d1e-8111-dec58dc61c55 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:17.206599 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9b99c-f609-4d1e-8111-dec58dc61c55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:17.212202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c9b99c-f609-4d1e-8111-dec58dc61c55 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:17.275608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 868/1734] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:16 2022] POST /identity/v3/users => generated 285 bytes in 573 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:17.278918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9b99c-f609-4d1e-8111-dec58dc61c55 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:17.328576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c9b99c-f609-4d1e-8111-dec58dc61c55 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:17.355859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0f4b4-7b3b-4eb5-b2eb-76440d2ae628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:17.357261 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0f4b4-7b3b-4eb5-b2eb-76440d2ae628 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:17.358752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0f4b4-7b3b-4eb5-b2eb-76440d2ae628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:17.359048 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0f4b4-7b3b-4eb5-b2eb-76440d2ae628 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:17.360715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0f4b4-7b3b-4eb5-b2eb-76440d2ae628 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:17.367843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0f4b4-7b3b-4eb5-b2eb-76440d2ae628 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:17.371693 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ae11bc-16e3-414c-837e-c570abcd1b13 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:17.377989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da816b9-fd03-4b93-82a1-c78cf1b1f9b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:17.379342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da816b9-fd03-4b93-82a1-c78cf1b1f9b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:17.379807 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da816b9-fd03-4b93-82a1-c78cf1b1f9b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:17.380258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da816b9-fd03-4b93-82a1-c78cf1b1f9b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:17.381115 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da816b9-fd03-4b93-82a1-c78cf1b1f9b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:17.383423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da816b9-fd03-4b93-82a1-c78cf1b1f9b1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:17.410552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 869/1735] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:17.412353 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0f4b4-7b3b-4eb5-b2eb-76440d2ae628 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:17.428108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 867/1736] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:17 2022] POST /identity/v3/projects => generated 377 bytes in 315 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:17.433474 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da816b9-fd03-4b93-82a1-c78cf1b1f9b1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:17.434598 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0f4b4-7b3b-4eb5-b2eb-76440d2ae628 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:17.553935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da816b9-fd03-4b93-82a1-c78cf1b1f9b1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:17.586554 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c131aa7-354b-4ab8-afb1-64039058fafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:17.592146 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c131aa7-354b-4ab8-afb1-64039058fafe admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:17.592449 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c131aa7-354b-4ab8-afb1-64039058fafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:17.596470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c131aa7-354b-4ab8-afb1-64039058fafe admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:17.597377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c131aa7-354b-4ab8-afb1-64039058fafe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:17.611540 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c131aa7-354b-4ab8-afb1-64039058fafe admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:17.692812 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 868/1737] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:17 2022] POST /identity/v3/users => generated 289 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:17.694531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5e2be-bc3a-4bbe-9a04-7973eddc1ff2 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:17.694831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5e2be-bc3a-4bbe-9a04-7973eddc1ff2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:17.695185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae5e2be-bc3a-4bbe-9a04-7973eddc1ff2 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:17.699544 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 869/1738] 10.222.0.85 () {54 vars in 921 bytes} [Wed May 25 16:08:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:17.704807 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5e2be-bc3a-4bbe-9a04-7973eddc1ff2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:17.707657 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198405db-d9ae-474c-8cd0-b4cb5d289c1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:17.708435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198405db-d9ae-474c-8cd0-b4cb5d289c1f admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:17.710015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198405db-d9ae-474c-8cd0-b4cb5d289c1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:17.710399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-198405db-d9ae-474c-8cd0-b4cb5d289c1f admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:17.715375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198405db-d9ae-474c-8cd0-b4cb5d289c1f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:17.727082 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-198405db-d9ae-474c-8cd0-b4cb5d289c1f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:17.736531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae5e2be-bc3a-4bbe-9a04-7973eddc1ff2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:17.748052 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db038734-c3c2-44eb-9ca5-3f8f978171eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:17.749146 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db038734-c3c2-44eb-9ca5-3f8f978171eb admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:17.750060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db038734-c3c2-44eb-9ca5-3f8f978171eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:17.751558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db038734-c3c2-44eb-9ca5-3f8f978171eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:17.755026 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db038734-c3c2-44eb-9ca5-3f8f978171eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:17.756959 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db038734-c3c2-44eb-9ca5-3f8f978171eb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:17.783976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 870/1739] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:17.787113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db038734-c3c2-44eb-9ca5-3f8f978171eb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:17.808945 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 870/1740] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 395 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:17.811591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db038734-c3c2-44eb-9ca5-3f8f978171eb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:17.819110 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198405db-d9ae-474c-8cd0-b4cb5d289c1f admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:17.824636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80b744-0dae-493e-a916-9248dc672492 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:17.827101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80b744-0dae-493e-a916-9248dc672492 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:17.827432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80b744-0dae-493e-a916-9248dc672492 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:17.827727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe80b744-0dae-493e-a916-9248dc672492 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:17.830196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80b744-0dae-493e-a916-9248dc672492 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:17.831922 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe80b744-0dae-493e-a916-9248dc672492 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:17.867026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-198405db-d9ae-474c-8cd0-b4cb5d289c1f admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:17.868633 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 871/1741] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:17 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:17.872070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80b744-0dae-493e-a916-9248dc672492 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:17.885739 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3108d9d9-0502-4bde-89cf-872fb47d5dc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:17.892568 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108d9d9-0502-4bde-89cf-872fb47d5dc2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:17.893218 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108d9d9-0502-4bde-89cf-872fb47d5dc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:17.893829 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3108d9d9-0502-4bde-89cf-872fb47d5dc2 admin admin] PATH_INFO: `/v3/projects/a987e327d3654db980fd0acbe2faed81/users/616e514cc82244e8a9674497ca177291/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:17.897907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe80b744-0dae-493e-a916-9248dc672492 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:17.913974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcad59f3-474a-434f-92d7-efd3d02e5525 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:17.914864 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad59f3-474a-434f-92d7-efd3d02e5525 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:17.915151 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad59f3-474a-434f-92d7-efd3d02e5525 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:17.915645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcad59f3-474a-434f-92d7-efd3d02e5525 admin admin] PATH_INFO: `/v3/projects/0ba8ffbaba534893bede4661d877beda/users/7ca575d2be31474091996544b5498889/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:17.927572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108d9d9-0502-4bde-89cf-872fb47d5dc2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a987e327d3654db980fd0acbe2faed81, user_id=616e514cc82244e8a9674497ca177291, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:17.938706 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3108d9d9-0502-4bde-89cf-872fb47d5dc2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:17.960969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad59f3-474a-434f-92d7-efd3d02e5525 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0ba8ffbaba534893bede4661d877beda, user_id=7ca575d2be31474091996544b5498889, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:17.977808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 871/1742] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:17 2022] PUT /identity/v3/projects/a987e327d3654db980fd0acbe2faed81/users/616e514cc82244e8a9674497ca177291/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:17.978015 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcad59f3-474a-434f-92d7-efd3d02e5525 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:17.978113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bac53-cad1-46ea-a192-184f9b860bc6 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:17.978113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bac53-cad1-46ea-a192-184f9b860bc6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:17.978113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a5bac53-cad1-46ea-a192-184f9b860bc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:18.021807 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 872/1743] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:17 2022] PUT /identity/v3/projects/0ba8ffbaba534893bede4661d877beda/users/7ca575d2be31474091996544b5498889/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:18.026243 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87c70c-5314-4d36-bf12-f8db9f99f054 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:18.026543 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87c70c-5314-4d36-bf12-f8db9f99f054 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:18.026836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c87c70c-5314-4d36-bf12-f8db9f99f054 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:18.033291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-1a5bac53-cad1-46ea-a192-184f9b860bc6 None None] MFA Rules not processed for user `4751abc9f67947b3ad17ba2b78ce2c12`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:18.041186 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1a5bac53-cad1-46ea-a192-184f9b860bc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:18.139569 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6c87c70c-5314-4d36-bf12-f8db9f99f054 None None] MFA Rules not processed for user `616e514cc82244e8a9674497ca177291`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:18.141171 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6c87c70c-5314-4d36-bf12-f8db9f99f054 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:18.219621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:18.225826 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 872/1744] 10.222.0.85 () {58 vars in 1018 bytes} [Wed May 25 16:08:17 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 256 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:18.228754 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6f9b3-8b73-41c7-914c-57e4605acd2f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:18.229251 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6f9b3-8b73-41c7-914c-57e4605acd2f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:18.229741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd6f9b3-8b73-41c7-914c-57e4605acd2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:18.269390 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-0fd6f9b3-8b73-41c7-914c-57e4605acd2f None None] MFA Rules not processed for user `7ca575d2be31474091996544b5498889`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:18.271046 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0fd6f9b3-8b73-41c7-914c-57e4605acd2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:18.286599 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:18.349412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 873/1745] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:18 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 327 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:18.353962 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87c70c-5314-4d36-bf12-f8db9f99f054 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:18.432699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 873/1746] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:18 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:18.439492 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6f9b3-8b73-41c7-914c-57e4605acd2f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:18.509618 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c87c70c-5314-4d36-bf12-f8db9f99f054 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:18.523303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339293a-e332-4a49-a5fc-8cb908794230 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:18.524079 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339293a-e332-4a49-a5fc-8cb908794230 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:18.524371 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339293a-e332-4a49-a5fc-8cb908794230 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:18.526511 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4339293a-e332-4a49-a5fc-8cb908794230 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:18.527336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339293a-e332-4a49-a5fc-8cb908794230 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:18.534863 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4339293a-e332-4a49-a5fc-8cb908794230 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:18.598516 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd6f9b3-8b73-41c7-914c-57e4605acd2f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:18.602917 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 874/1747] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:18 2022] POST /identity/v3/projects => generated 367 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:18.603991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339293a-e332-4a49-a5fc-8cb908794230 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:18.607236 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4339293a-e332-4a49-a5fc-8cb908794230 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:18.810260 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eca055-81ca-441f-98f5-511c834845d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:18.812090 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eca055-81ca-441f-98f5-511c834845d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:18.812453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eca055-81ca-441f-98f5-511c834845d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:18.812732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96eca055-81ca-441f-98f5-511c834845d5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:18.817244 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eca055-81ca-441f-98f5-511c834845d5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:18.841610 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96eca055-81ca-441f-98f5-511c834845d5 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:18.904536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4339293a-e332-4a49-a5fc-8cb908794230 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:18.928391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c6c4d-3e7e-455c-b799-a6eb6ddd65ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:18.932042 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c6c4d-3e7e-455c-b799-a6eb6ddd65ba service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:18.932378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c6c4d-3e7e-455c-b799-a6eb6ddd65ba service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:18.932676 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8c6c4d-3e7e-455c-b799-a6eb6ddd65ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:18.972834 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c6c4d-3e7e-455c-b799-a6eb6ddd65ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:18.975378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8c6c4d-3e7e-455c-b799-a6eb6ddd65ba service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:19.007894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 874/1748] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:18 2022] POST /identity/v3/projects => generated 377 bytes in 566 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:19.009863 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eca055-81ca-441f-98f5-511c834845d5 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:19.027090 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 875/1749] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:18 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 424 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:19.043675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c6c4d-3e7e-455c-b799-a6eb6ddd65ba service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:19.085621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8c6c4d-3e7e-455c-b799-a6eb6ddd65ba service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:19.256481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:19.279056 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:19.315123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96eca055-81ca-441f-98f5-511c834845d5 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:19.337808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7b9f2-aa12-41bc-8095-4e948833290e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:19.339719 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7b9f2-aa12-41bc-8095-4e948833290e admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:19.341655 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7b9f2-aa12-41bc-8095-4e948833290e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:19.343279 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf7b9f2-aa12-41bc-8095-4e948833290e admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:19.344226 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf7b9f2-aa12-41bc-8095-4e948833290e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:19.349027 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59fcdd5-00d3-448c-a1fc-ca4c3ec7cebc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:19.357023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fcdd5-00d3-448c-a1fc-ca4c3ec7cebc admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:19.357023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fcdd5-00d3-448c-a1fc-ca4c3ec7cebc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:19.357023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59fcdd5-00d3-448c-a1fc-ca4c3ec7cebc admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:19.357023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59fcdd5-00d3-448c-a1fc-ca4c3ec7cebc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:19.357411 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf7b9f2-aa12-41bc-8095-4e948833290e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:19.357510 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59fcdd5-00d3-448c-a1fc-ca4c3ec7cebc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:19.419573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 876/1750] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:19 2022] POST /identity/v3/users => generated 295 bytes in 375 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:19.423872 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 875/1751] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:19 2022] POST /identity/v3/users => generated 290 bytes in 415 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:19.424048 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c22f2-952a-4a5f-8708-a872043e4ba2 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:19.424163 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7b9f2-aa12-41bc-8095-4e948833290e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:19.424256 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c22f2-952a-4a5f-8708-a872043e4ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:19.425484 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75c22f2-952a-4a5f-8708-a872043e4ba2 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:19.430905 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 877/1752] 10.222.0.85 () {54 vars in 921 bytes} [Wed May 25 16:08:19 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:19.433082 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c22f2-952a-4a5f-8708-a872043e4ba2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:19.465003 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf7b9f2-aa12-41bc-8095-4e948833290e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:19.473763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75c22f2-952a-4a5f-8708-a872043e4ba2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:19.490578 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce168b-9441-4b6d-af45-e1b0f8785969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:19.491283 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce168b-9441-4b6d-af45-e1b0f8785969 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:19.491578 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce168b-9441-4b6d-af45-e1b0f8785969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:19.494556 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ce168b-9441-4b6d-af45-e1b0f8785969 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:19.495463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce168b-9441-4b6d-af45-e1b0f8785969 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:19.496346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27e365-0d41-4ab5-9af4-5e2ae71da363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:19.497163 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27e365-0d41-4ab5-9af4-5e2ae71da363 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:19.497293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73ce168b-9441-4b6d-af45-e1b0f8785969 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:19.509411 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27e365-0d41-4ab5-9af4-5e2ae71da363 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:19.509884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee27e365-0d41-4ab5-9af4-5e2ae71da363 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:19.510800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27e365-0d41-4ab5-9af4-5e2ae71da363 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:19.512795 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee27e365-0d41-4ab5-9af4-5e2ae71da363 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:19.537181 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 878/1753] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 106 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:19.538226 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce168b-9441-4b6d-af45-e1b0f8785969 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:19.546572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 876/1754] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:19.554956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27e365-0d41-4ab5-9af4-5e2ae71da363 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:19.567805 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ce168b-9441-4b6d-af45-e1b0f8785969 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:19.585152 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4253dab-b468-4248-92f1-80d5aeb20f68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:19.586236 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4253dab-b468-4248-92f1-80d5aeb20f68 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:19.586527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4253dab-b468-4248-92f1-80d5aeb20f68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:19.586817 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4253dab-b468-4248-92f1-80d5aeb20f68 admin admin] PATH_INFO: `/v3/projects/db3bf79857a74695971d4f1f5e92ef60/users/f2895da85d9d43c08bd41d2a729eb002/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:19.591247 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee27e365-0d41-4ab5-9af4-5e2ae71da363 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:19.609005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aebb9b-3929-473f-9062-ea65dc2c4c2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:19.610064 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aebb9b-3929-473f-9062-ea65dc2c4c2a admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:19.610464 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aebb9b-3929-473f-9062-ea65dc2c4c2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:19.612365 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21aebb9b-3929-473f-9062-ea65dc2c4c2a admin admin] PATH_INFO: `/v3/projects/4ba513ec36cb4a43a511c08727c83f83/users/91105958bb0242838cdb2f0e213c674e/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:19.628152 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4253dab-b468-4248-92f1-80d5aeb20f68 admin admin] RBAC: Authorizing `identity:create_grant(project_id=db3bf79857a74695971d4f1f5e92ef60, user_id=f2895da85d9d43c08bd41d2a729eb002, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:19.637241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4253dab-b468-4248-92f1-80d5aeb20f68 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:19.664068 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aebb9b-3929-473f-9062-ea65dc2c4c2a admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ba513ec36cb4a43a511c08727c83f83, user_id=91105958bb0242838cdb2f0e213c674e, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:19.671168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21aebb9b-3929-473f-9062-ea65dc2c4c2a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:19.675680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 879/1755] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:19 2022] PUT /identity/v3/projects/db3bf79857a74695971d4f1f5e92ef60/users/f2895da85d9d43c08bd41d2a729eb002/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:19.684980 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4253dab-b468-4248-92f1-80d5aeb20f68 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:19.721422 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 877/1756] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:19 2022] PUT /identity/v3/projects/4ba513ec36cb4a43a511c08727c83f83/users/91105958bb0242838cdb2f0e213c674e/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:19.725530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aebb9b-3929-473f-9062-ea65dc2c4c2a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:19.915416 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4253dab-b468-4248-92f1-80d5aeb20f68 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:19.956265 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21aebb9b-3929-473f-9062-ea65dc2c4c2a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:19.972369 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf93c8a-7496-4315-8c45-787ca85f0bc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:19.973497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf93c8a-7496-4315-8c45-787ca85f0bc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:19.974043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf93c8a-7496-4315-8c45-787ca85f0bc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:19.974577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf93c8a-7496-4315-8c45-787ca85f0bc8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:19.977235 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf93c8a-7496-4315-8c45-787ca85f0bc8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:19.981224 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddf93c8a-7496-4315-8c45-787ca85f0bc8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:20.019274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 878/1757] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:20.020958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7c1c4-c579-4a32-a998-7911bbff75dd None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:20.022159 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7c1c4-c579-4a32-a998-7911bbff75dd None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:20.022159 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9a7c1c4-c579-4a32-a998-7911bbff75dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:20.077505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c9a7c1c4-c579-4a32-a998-7911bbff75dd None None] MFA Rules not processed for user `4751abc9f67947b3ad17ba2b78ce2c12`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:20.078799 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c9a7c1c4-c579-4a32-a998-7911bbff75dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:20.085466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a2d45-db35-4c11-907e-2d8d8b13cc3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:20.093344 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a2d45-db35-4c11-907e-2d8d8b13cc3a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:20.093744 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a2d45-db35-4c11-907e-2d8d8b13cc3a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:20.094439 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b72a2d45-db35-4c11-907e-2d8d8b13cc3a admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:20.095566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a2d45-db35-4c11-907e-2d8d8b13cc3a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:20.097643 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b72a2d45-db35-4c11-907e-2d8d8b13cc3a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:20.127345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 880/1758] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:19 2022] GET /identity/v3/roles => generated 1420 bytes in 443 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:20.133284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a2d45-db35-4c11-907e-2d8d8b13cc3a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:20.138412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b72a2d45-db35-4c11-907e-2d8d8b13cc3a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:20.166923 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b72a2d45-db35-4c11-907e-2d8d8b13cc3a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:20.181249 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb77121f-5300-4782-a6be-446a5f4591f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:20.182186 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77121f-5300-4782-a6be-446a5f4591f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:20.182488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77121f-5300-4782-a6be-446a5f4591f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:20.182774 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb77121f-5300-4782-a6be-446a5f4591f6 admin admin] PATH_INFO: `/v3/users/dfa1bae9ee9f47e1b330391b90bc4fd1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:20.186436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77121f-5300-4782-a6be-446a5f4591f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dfa1bae9ee9f47e1b330391b90bc4fd1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:20.192772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb77121f-5300-4782-a6be-446a5f4591f6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:20.231715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 879/1759] 10.222.0.85 () {58 vars in 1018 bytes} [Wed May 25 16:08:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:20.232900 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7c1c4-c579-4a32-a998-7911bbff75dd None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:20.247880 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9a7c1c4-c579-4a32-a998-7911bbff75dd None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:20.257817 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e3d78-802f-4bd7-bab5-6fcdb05ac3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:20.258770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e3d78-802f-4bd7-bab5-6fcdb05ac3f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:20.259062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e3d78-802f-4bd7-bab5-6fcdb05ac3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:20.260693 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233e3d78-802f-4bd7-bab5-6fcdb05ac3f4 admin admin] PATH_INFO: `/v3/projects/4ba513ec36cb4a43a511c08727c83f83/users/91105958bb0242838cdb2f0e213c674e/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:20.267217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e3d78-802f-4bd7-bab5-6fcdb05ac3f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ba513ec36cb4a43a511c08727c83f83, user_id=91105958bb0242838cdb2f0e213c674e, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:20.271916 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233e3d78-802f-4bd7-bab5-6fcdb05ac3f4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:20.286799 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 880/1760] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:20 2022] PUT /identity/v3/projects/4ba513ec36cb4a43a511c08727c83f83/users/91105958bb0242838cdb2f0e213c674e/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:20.287857 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e3d78-802f-4bd7-bab5-6fcdb05ac3f4 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:20.318466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:20.336601 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:INSERT=4,keystone:DELETE=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:20.363943 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-eb77121f-5300-4782-a6be-446a5f4591f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dfa1bae9ee9f47e1b330391b90bc4fd1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:20.396116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-eb77121f-5300-4782-a6be-446a5f4591f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dfa1bae9ee9f47e1b330391b90bc4fd1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:20.427332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-eb77121f-5300-4782-a6be-446a5f4591f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dfa1bae9ee9f47e1b330391b90bc4fd1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:20.428170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-eb77121f-5300-4782-a6be-446a5f4591f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dfa1bae9ee9f47e1b330391b90bc4fd1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:20.440458 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-eb77121f-5300-4782-a6be-446a5f4591f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dfa1bae9ee9f47e1b330391b90bc4fd1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:20.459549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 881/1761] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:20 2022] DELETE /identity/v3/users/dfa1bae9ee9f47e1b330391b90bc4fd1 => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:20.459549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1094943-6591-43cb-8e2e-3ee989d37852 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:20.459549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1094943-6591-43cb-8e2e-3ee989d37852 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:20.459549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1094943-6591-43cb-8e2e-3ee989d37852 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:20.483814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233e3d78-802f-4bd7-bab5-6fcdb05ac3f4 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:20.518828 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d1094943-6591-43cb-8e2e-3ee989d37852 None None] MFA Rules not processed for user `91105958bb0242838cdb2f0e213c674e`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:20.523241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d1094943-6591-43cb-8e2e-3ee989d37852 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:20.649700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eddf41b-516b-46f9-a25a-43d9b2b9ef1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:20.652601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddf41b-516b-46f9-a25a-43d9b2b9ef1c admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:20.652899 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddf41b-516b-46f9-a25a-43d9b2b9ef1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:20.653192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eddf41b-516b-46f9-a25a-43d9b2b9ef1c admin admin] PATH_INFO: `/v3/projects/db3bf79857a74695971d4f1f5e92ef60/users/f2895da85d9d43c08bd41d2a729eb002/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:20.660174 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddf41b-516b-46f9-a25a-43d9b2b9ef1c admin admin] RBAC: Authorizing `identity:create_grant(project_id=db3bf79857a74695971d4f1f5e92ef60, user_id=f2895da85d9d43c08bd41d2a729eb002, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:20.667697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eddf41b-516b-46f9-a25a-43d9b2b9ef1c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:20.706072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 881/1762] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:20 2022] PUT /identity/v3/projects/db3bf79857a74695971d4f1f5e92ef60/users/f2895da85d9d43c08bd41d2a729eb002/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 419 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:20.707485 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcb962-a9ed-44d6-83d0-7ffc8bb056cc None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:20.707782 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcb962-a9ed-44d6-83d0-7ffc8bb056cc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:20.708095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89dcb962-a9ed-44d6-83d0-7ffc8bb056cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:20.722790 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 882/1763] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:20 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 261 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:20.722790 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3e598-aa48-4f10-b0c9-9a0d69250431 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:20.722790 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3e598-aa48-4f10-b0c9-9a0d69250431 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:20.722790 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3e598-aa48-4f10-b0c9-9a0d69250431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:20.755628 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-89dcb962-a9ed-44d6-83d0-7ffc8bb056cc None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:20.756807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-89dcb962-a9ed-44d6-83d0-7ffc8bb056cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:20.762488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7cc3e598-aa48-4f10-b0c9-9a0d69250431 None None] MFA Rules not processed for user `f2895da85d9d43c08bd41d2a729eb002`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:20.764132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7cc3e598-aa48-4f10-b0c9-9a0d69250431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:20.890332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 883/1764] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:20 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:20.895771 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3e598-aa48-4f10-b0c9-9a0d69250431 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:20.897297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7cc3e598-aa48-4f10-b0c9-9a0d69250431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:20.904936 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 882/1765] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:20.910361 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dcb962-a9ed-44d6-83d0-7ffc8bb056cc None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:20.912014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-89dcb962-a9ed-44d6-83d0-7ffc8bb056cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:20.930845 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:20.972942 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3e598-aa48-4f10-b0c9-9a0d69250431 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:20.980027 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa5000-3740-4c48-88e8-1d158b407353 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:20.980715 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa5000-3740-4c48-88e8-1d158b407353 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:20.981016 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa5000-3740-4c48-88e8-1d158b407353 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:20.981622 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81fa5000-3740-4c48-88e8-1d158b407353 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:20.991448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:21.026564 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89dcb962-a9ed-44d6-83d0-7ffc8bb056cc None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:21.034268 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1365a16-db8c-48f1-8bae-f9c59f0c03bc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:21.035047 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1365a16-db8c-48f1-8bae-f9c59f0c03bc service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:21.035398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1365a16-db8c-48f1-8bae-f9c59f0c03bc service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:21.035740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1365a16-db8c-48f1-8bae-f9c59f0c03bc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:21.038038 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d1365a16-db8c-48f1-8bae-f9c59f0c03bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:21.052641 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1365a16-db8c-48f1-8bae-f9c59f0c03bc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:21.054983 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1365a16-db8c-48f1-8bae-f9c59f0c03bc service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:21.069173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 883/1766] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 160 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:21.077004 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa5000-3740-4c48-88e8-1d158b407353 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:21.078676 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81fa5000-3740-4c48-88e8-1d158b407353 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:21.084134 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1365a16-db8c-48f1-8bae-f9c59f0c03bc service nova] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:21.092576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d1365a16-db8c-48f1-8bae-f9c59f0c03bc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:21.101926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 884/1767] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:20 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 211 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:21.103476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa5000-3740-4c48-88e8-1d158b407353 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:21.109238 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-81fa5000-3740-4c48-88e8-1d158b407353 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:21.117064 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1365a16-db8c-48f1-8bae-f9c59f0c03bc service nova] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:21.126008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e37b9-be5d-46ac-8fb3-98caf33ce3f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:21.127327 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e37b9-be5d-46ac-8fb3-98caf33ce3f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:21.127737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e37b9-be5d-46ac-8fb3-98caf33ce3f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:21.128469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8e37b9-be5d-46ac-8fb3-98caf33ce3f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:21.154480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e37b9-be5d-46ac-8fb3-98caf33ce3f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:21.158413 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8e37b9-be5d-46ac-8fb3-98caf33ce3f5 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:21.190089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 884/1768] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:21 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:21.191712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e37b9-be5d-46ac-8fb3-98caf33ce3f5 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:21.214538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8e37b9-be5d-46ac-8fb3-98caf33ce3f5 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:21.224592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea4600-edb1-449d-b487-bde0dfcfcec4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:21.228298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea4600-edb1-449d-b487-bde0dfcfcec4 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:21.229535 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea4600-edb1-449d-b487-bde0dfcfcec4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:21.230076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ea4600-edb1-449d-b487-bde0dfcfcec4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:21.236343 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81fa5000-3740-4c48-88e8-1d158b407353 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:21.237111 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-36ea4600-edb1-449d-b487-bde0dfcfcec4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:21.240291 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be3aaa5-d5ab-4386-bda6-6e5ac55a73bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:21.241119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3aaa5-d5ab-4386-bda6-6e5ac55a73bf service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:21.241587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3aaa5-d5ab-4386-bda6-6e5ac55a73bf service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:21.241890 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be3aaa5-d5ab-4386-bda6-6e5ac55a73bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:21.264715 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3aaa5-d5ab-4386-bda6-6e5ac55a73bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:21.266735 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be3aaa5-d5ab-4386-bda6-6e5ac55a73bf service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:21.280360 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 885/1769] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:21.348158 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea4600-edb1-449d-b487-bde0dfcfcec4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:21.349644 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ea4600-edb1-449d-b487-bde0dfcfcec4 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:21.370681 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 885/1770] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:21 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 180 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:21.958513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1c9da-74a6-4bf4-8b02-412e46cea1dd None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:21.959035 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1c9da-74a6-4bf4-8b02-412e46cea1dd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:21.959133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa1c9da-74a6-4bf4-8b02-412e46cea1dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:22.021009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dfa1c9da-74a6-4bf4-8b02-412e46cea1dd None None] MFA Rules not processed for user `708fdc537e1747d7bd5917ed7af9c368`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:22.022223 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dfa1c9da-74a6-4bf4-8b02-412e46cea1dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:22.230024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 886/1771] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:23.157816 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea4600-edb1-449d-b487-bde0dfcfcec4 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:23.159939 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-36ea4600-edb1-449d-b487-bde0dfcfcec4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:23.197162 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ea4600-edb1-449d-b487-bde0dfcfcec4 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:23.228560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f85711-0be4-49d3-8b03-8d1994e94f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:23.229668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f85711-0be4-49d3-8b03-8d1994e94f8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:23.230101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f85711-0be4-49d3-8b03-8d1994e94f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:23.230557 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f85711-0be4-49d3-8b03-8d1994e94f8c admin admin] PATH_INFO: `/v3/projects/30f8d2e80d6342ca90f689fbb3b78d5b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:23.233338 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f85711-0be4-49d3-8b03-8d1994e94f8c admin admin] RBAC: Authorizing `identity:delete_project(project_id=30f8d2e80d6342ca90f689fbb3b78d5b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:23.235447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f85711-0be4-49d3-8b03-8d1994e94f8c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:23.332413 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e7f85711-0be4-49d3-8b03-8d1994e94f8c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30f8d2e80d6342ca90f689fbb3b78d5b', 'request_id': 'req-e7f85711-0be4-49d3-8b03-8d1994e94f8c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:23.350647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e7f85711-0be4-49d3-8b03-8d1994e94f8c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30f8d2e80d6342ca90f689fbb3b78d5b', 'request_id': 'req-e7f85711-0be4-49d3-8b03-8d1994e94f8c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:23.367410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e7f85711-0be4-49d3-8b03-8d1994e94f8c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30f8d2e80d6342ca90f689fbb3b78d5b', 'request_id': 'req-e7f85711-0be4-49d3-8b03-8d1994e94f8c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:23.388148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e7f85711-0be4-49d3-8b03-8d1994e94f8c admin admin] The token cache is being invalidate because project 30f8d2e80d6342ca90f689fbb3b78d5b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:23.388715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e7f85711-0be4-49d3-8b03-8d1994e94f8c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:23.389569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e7f85711-0be4-49d3-8b03-8d1994e94f8c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:23.391581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 886/1772] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:23 2022] DELETE /identity/v3/projects/30f8d2e80d6342ca90f689fbb3b78d5b => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:23.396386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1c9da-74a6-4bf4-8b02-412e46cea1dd None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:23.417293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa1c9da-74a6-4bf4-8b02-412e46cea1dd None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:23.463746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:23.550806 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49bc53-97c4-4916-922e-5fb9926bede0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:23.552411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49bc53-97c4-4916-922e-5fb9926bede0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:23.553006 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49bc53-97c4-4916-922e-5fb9926bede0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:23.553667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef49bc53-97c4-4916-922e-5fb9926bede0 admin admin] PATH_INFO: `/v3/users/3c938ac58c1d4cb09e76b9f6fd61c0d8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:23.556309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49bc53-97c4-4916-922e-5fb9926bede0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c938ac58c1d4cb09e76b9f6fd61c0d8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:23.559729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef49bc53-97c4-4916-922e-5fb9926bede0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:23.666792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ef49bc53-97c4-4916-922e-5fb9926bede0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c938ac58c1d4cb09e76b9f6fd61c0d8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:23.699456 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ef49bc53-97c4-4916-922e-5fb9926bede0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c938ac58c1d4cb09e76b9f6fd61c0d8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:23.726715 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ef49bc53-97c4-4916-922e-5fb9926bede0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c938ac58c1d4cb09e76b9f6fd61c0d8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:23.728385 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ef49bc53-97c4-4916-922e-5fb9926bede0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c938ac58c1d4cb09e76b9f6fd61c0d8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:23.759418 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ef49bc53-97c4-4916-922e-5fb9926bede0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c938ac58c1d4cb09e76b9f6fd61c0d8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:23.782735 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 887/1773] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:23 2022] DELETE /identity/v3/users/3c938ac58c1d4cb09e76b9f6fd61c0d8 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:23.871780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b52461-614d-41d9-b44a-202a23c86c8d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:23.872459 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b52461-614d-41d9-b44a-202a23c86c8d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:23.873066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b52461-614d-41d9-b44a-202a23c86c8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:23.880556 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49bc53-97c4-4916-922e-5fb9926bede0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:23.882413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ef49bc53-97c4-4916-922e-5fb9926bede0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:23.931366 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-a6b52461-614d-41d9-b44a-202a23c86c8d None None] MFA Rules not processed for user `a0289402c6f245839a86b7b66714a8f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:23.935414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a6b52461-614d-41d9-b44a-202a23c86c8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:23.976198 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef49bc53-97c4-4916-922e-5fb9926bede0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:23.981720 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb833eac-f877-419d-856a-13000b6b54a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:23.983015 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb833eac-f877-419d-856a-13000b6b54a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:23.983417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb833eac-f877-419d-856a-13000b6b54a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:23.983795 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb833eac-f877-419d-856a-13000b6b54a3 admin admin] PATH_INFO: `/v3/users/616e514cc82244e8a9674497ca177291` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:23.985267 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb833eac-f877-419d-856a-13000b6b54a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=616e514cc82244e8a9674497ca177291)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:23.987483 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb833eac-f877-419d-856a-13000b6b54a3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:24.054205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:24.061054 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb833eac-f877-419d-856a-13000b6b54a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '616e514cc82244e8a9674497ca177291'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:24.079319 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb833eac-f877-419d-856a-13000b6b54a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '616e514cc82244e8a9674497ca177291'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:24.080014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:24.091470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb833eac-f877-419d-856a-13000b6b54a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '616e514cc82244e8a9674497ca177291'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:24.092103 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb833eac-f877-419d-856a-13000b6b54a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '616e514cc82244e8a9674497ca177291'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:24.102023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb833eac-f877-419d-856a-13000b6b54a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '616e514cc82244e8a9674497ca177291'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:24.105565 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 887/1774] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:24.136375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 888/1775] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:23 2022] DELETE /identity/v3/users/616e514cc82244e8a9674497ca177291 => generated 0 bytes in 254 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:24.141582 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b07723-8d5f-4052-b75a-b38adafb565e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:24.141953 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b07723-8d5f-4052-b75a-b38adafb565e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:24.143909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b07723-8d5f-4052-b75a-b38adafb565e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:24.209925 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-51b07723-8d5f-4052-b75a-b38adafb565e None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:24.211412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-51b07723-8d5f-4052-b75a-b38adafb565e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:24.290620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 888/1776] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:24.336598 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb833eac-f877-419d-856a-13000b6b54a3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:24.338125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cb833eac-f877-419d-856a-13000b6b54a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:24.447166 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb833eac-f877-419d-856a-13000b6b54a3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:24.452460 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250abb82-a54a-4c47-b5df-56f6b95292dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:24.453531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250abb82-a54a-4c47-b5df-56f6b95292dc service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:24.454375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250abb82-a54a-4c47-b5df-56f6b95292dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:24.454973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250abb82-a54a-4c47-b5df-56f6b95292dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:24.469196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250abb82-a54a-4c47-b5df-56f6b95292dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:24.471273 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250abb82-a54a-4c47-b5df-56f6b95292dc service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:24.485664 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 889/1777] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:25.209335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b07723-8d5f-4052-b75a-b38adafb565e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:25.210871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-51b07723-8d5f-4052-b75a-b38adafb565e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:25.224805 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b07723-8d5f-4052-b75a-b38adafb565e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:25.243485 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e80e219-e07c-4760-8f83-48fc7252a94b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:25.249799 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80e219-e07c-4760-8f83-48fc7252a94b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:25.250129 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80e219-e07c-4760-8f83-48fc7252a94b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:25.250419 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e80e219-e07c-4760-8f83-48fc7252a94b admin admin] PATH_INFO: `/v3/users/7605790928584006a2da23db8afb2b95` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:25.252484 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80e219-e07c-4760-8f83-48fc7252a94b admin admin] RBAC: Authorizing `identity:delete_user(user_id=7605790928584006a2da23db8afb2b95)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:25.254652 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e80e219-e07c-4760-8f83-48fc7252a94b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:25.354812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5e80e219-e07c-4760-8f83-48fc7252a94b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7605790928584006a2da23db8afb2b95'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:25.355522 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5e80e219-e07c-4760-8f83-48fc7252a94b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7605790928584006a2da23db8afb2b95'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:25.379190 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5e80e219-e07c-4760-8f83-48fc7252a94b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7605790928584006a2da23db8afb2b95'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:25.404097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5e80e219-e07c-4760-8f83-48fc7252a94b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7605790928584006a2da23db8afb2b95'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:25.405581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5e80e219-e07c-4760-8f83-48fc7252a94b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7605790928584006a2da23db8afb2b95'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:25.495918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 889/1778] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:25 2022] DELETE /identity/v3/users/7605790928584006a2da23db8afb2b95 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:25.497382 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc928019-04bd-4ec4-8581-f1681bd62076 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:25.498181 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc928019-04bd-4ec4-8581-f1681bd62076 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:25.498609 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc928019-04bd-4ec4-8581-f1681bd62076 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:25.529979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bc928019-04bd-4ec4-8581-f1681bd62076 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:25.531480 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bc928019-04bd-4ec4-8581-f1681bd62076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:25.575663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a8133-59a6-441b-b3cb-877575427fd2 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:25.577061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a8133-59a6-441b-b3cb-877575427fd2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:25.577346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a8133-59a6-441b-b3cb-877575427fd2 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:25.579838 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 890/1779] 10.222.0.85 () {54 vars in 921 bytes} [Wed May 25 16:08:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:25.592405 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7707a-dea6-4515-8e5f-29aab1c4063e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:25.593288 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7707a-dea6-4515-8e5f-29aab1c4063e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:25.593646 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da7707a-dea6-4515-8e5f-29aab1c4063e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:25.661005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8da7707a-dea6-4515-8e5f-29aab1c4063e None None] MFA Rules not processed for user `4751abc9f67947b3ad17ba2b78ce2c12`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:25.664382 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8da7707a-dea6-4515-8e5f-29aab1c4063e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:25.667880 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 890/1780] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:08:25.729304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc928019-04bd-4ec4-8581-f1681bd62076 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:25.730776 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bc928019-04bd-4ec4-8581-f1681bd62076 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:25.797927 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:25.820953 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 891/1781] 10.222.0.85 () {58 vars in 1018 bytes} [Wed May 25 16:08:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:25.860671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc928019-04bd-4ec4-8581-f1681bd62076 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:25.871609 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a9e1b94-4b2f-4958-b78c-0c18b7f5667b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:25.872332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e1b94-4b2f-4958-b78c-0c18b7f5667b service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:25.872476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e1b94-4b2f-4958-b78c-0c18b7f5667b service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:25.872783 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a9e1b94-4b2f-4958-b78c-0c18b7f5667b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:25.888216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e1b94-4b2f-4958-b78c-0c18b7f5667b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:25.890968 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a9e1b94-4b2f-4958-b78c-0c18b7f5667b service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:25.928536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 891/1782] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:08:25.969441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7707a-dea6-4515-8e5f-29aab1c4063e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:25.970958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8da7707a-dea6-4515-8e5f-29aab1c4063e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:25.990831 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da7707a-dea6-4515-8e5f-29aab1c4063e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:25.995805 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52183818-2c2b-4365-bfd8-66e36c312f6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:25.996753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52183818-2c2b-4365-bfd8-66e36c312f6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:25.997046 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52183818-2c2b-4365-bfd8-66e36c312f6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:25.997327 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52183818-2c2b-4365-bfd8-66e36c312f6a admin admin] PATH_INFO: `/v3/projects/421601a3bf9b43c6914d2fea9e75e62f` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:26.002520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52183818-2c2b-4365-bfd8-66e36c312f6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=421601a3bf9b43c6914d2fea9e75e62f)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:26.004721 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52183818-2c2b-4365-bfd8-66e36c312f6a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:26.095098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-52183818-2c2b-4365-bfd8-66e36c312f6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '421601a3bf9b43c6914d2fea9e75e62f', 'request_id': 'req-52183818-2c2b-4365-bfd8-66e36c312f6a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:26.110718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-52183818-2c2b-4365-bfd8-66e36c312f6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '421601a3bf9b43c6914d2fea9e75e62f', 'request_id': 'req-52183818-2c2b-4365-bfd8-66e36c312f6a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:26.149742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-52183818-2c2b-4365-bfd8-66e36c312f6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '421601a3bf9b43c6914d2fea9e75e62f', 'request_id': 'req-52183818-2c2b-4365-bfd8-66e36c312f6a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:26.170501 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-52183818-2c2b-4365-bfd8-66e36c312f6a admin admin] The token cache is being invalidate because project 421601a3bf9b43c6914d2fea9e75e62f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:26.171227 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-52183818-2c2b-4365-bfd8-66e36c312f6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:26.172314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-52183818-2c2b-4365-bfd8-66e36c312f6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:26.175675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 892/1783] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:25 2022] DELETE /identity/v3/projects/421601a3bf9b43c6914d2fea9e75e62f => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:26.180907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee75ce-a47b-4b75-a0e2-f03851842ccd None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:26.181605 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee75ce-a47b-4b75-a0e2-f03851842ccd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:26.182043 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ee75ce-a47b-4b75-a0e2-f03851842ccd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:26.184665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52183818-2c2b-4365-bfd8-66e36c312f6a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:26.186945 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-52183818-2c2b-4365-bfd8-66e36c312f6a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:26.194170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:26.214936 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-47ee75ce-a47b-4b75-a0e2-f03851842ccd None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:26.216712 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-47ee75ce-a47b-4b75-a0e2-f03851842ccd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:26.317276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52183818-2c2b-4365-bfd8-66e36c312f6a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:26.328946 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6faef-4c01-4fcd-9210-846720dee7ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:26.329987 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6faef-4c01-4fcd-9210-846720dee7ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:26.331259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6faef-4c01-4fcd-9210-846720dee7ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:26.332246 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f6faef-4c01-4fcd-9210-846720dee7ac admin admin] PATH_INFO: `/v3/projects/a987e327d3654db980fd0acbe2faed81` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:26.336039 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6faef-4c01-4fcd-9210-846720dee7ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=a987e327d3654db980fd0acbe2faed81)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:26.339236 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f6faef-4c01-4fcd-9210-846720dee7ac admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:26.340779 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 892/1784] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:26.356302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91496418-0ef9-47ce-b08f-28aaafd191c8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:26.356600 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91496418-0ef9-47ce-b08f-28aaafd191c8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:26.357172 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91496418-0ef9-47ce-b08f-28aaafd191c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:26.398398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-91496418-0ef9-47ce-b08f-28aaafd191c8 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:26.399727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-91496418-0ef9-47ce-b08f-28aaafd191c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:26.413662 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 893/1785] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:26.415337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91496418-0ef9-47ce-b08f-28aaafd191c8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:26.417965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-91496418-0ef9-47ce-b08f-28aaafd191c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:26.428606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f3f6faef-4c01-4fcd-9210-846720dee7ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a987e327d3654db980fd0acbe2faed81', 'request_id': 'req-f3f6faef-4c01-4fcd-9210-846720dee7ac'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:26.436867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f3f6faef-4c01-4fcd-9210-846720dee7ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a987e327d3654db980fd0acbe2faed81', 'request_id': 'req-f3f6faef-4c01-4fcd-9210-846720dee7ac'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:26.465137 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f3f6faef-4c01-4fcd-9210-846720dee7ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a987e327d3654db980fd0acbe2faed81', 'request_id': 'req-f3f6faef-4c01-4fcd-9210-846720dee7ac'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:26.507220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f3f6faef-4c01-4fcd-9210-846720dee7ac admin admin] The token cache is being invalidate because project a987e327d3654db980fd0acbe2faed81 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:26.507589 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f3f6faef-4c01-4fcd-9210-846720dee7ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:26.511206 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f3f6faef-4c01-4fcd-9210-846720dee7ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:26.516431 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 893/1786] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:26 2022] DELETE /identity/v3/projects/a987e327d3654db980fd0acbe2faed81 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:26.516431 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6faef-4c01-4fcd-9210-846720dee7ac admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:26.519296 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f3f6faef-4c01-4fcd-9210-846720dee7ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:26.555095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:26.555361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91496418-0ef9-47ce-b08f-28aaafd191c8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:26.685351 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f6faef-4c01-4fcd-9210-846720dee7ac admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:26.692583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c4041-7b91-424f-b68f-0750743edff6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:26.693326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c4041-7b91-424f-b68f-0750743edff6 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:26.693565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c4041-7b91-424f-b68f-0750743edff6 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:26.693853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-941c4041-7b91-424f-b68f-0750743edff6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:26.701280 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-941c4041-7b91-424f-b68f-0750743edff6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:26.703019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dc3e6-6f42-4007-94fc-f4079e68aa5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:26.704249 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dc3e6-6f42-4007-94fc-f4079e68aa5d admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:26.704249 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dc3e6-6f42-4007-94fc-f4079e68aa5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:26.704408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295dc3e6-6f42-4007-94fc-f4079e68aa5d admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:26.705318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dc3e6-6f42-4007-94fc-f4079e68aa5d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:26.705645 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dc3e6-6f42-4007-94fc-f4079e68aa5d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:26.707303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295dc3e6-6f42-4007-94fc-f4079e68aa5d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:26.724693 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 894/1787] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:26.726148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dc3e6-6f42-4007-94fc-f4079e68aa5d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:26.727990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-295dc3e6-6f42-4007-94fc-f4079e68aa5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:26.733977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c4041-7b91-424f-b68f-0750743edff6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:26.735811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-941c4041-7b91-424f-b68f-0750743edff6 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:26.745599 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295dc3e6-6f42-4007-94fc-f4079e68aa5d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:26.751396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ce8ac-aa2f-4621-8133-8007a1b08e44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:26.752652 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ce8ac-aa2f-4621-8133-8007a1b08e44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:26.753144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ce8ac-aa2f-4621-8133-8007a1b08e44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:26.753662 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-095ce8ac-aa2f-4621-8133-8007a1b08e44 admin admin] PATH_INFO: `/v3/users/f2895da85d9d43c08bd41d2a729eb002` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:26.756034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ce8ac-aa2f-4621-8133-8007a1b08e44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2895da85d9d43c08bd41d2a729eb002)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:26.768787 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 894/1788] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:26 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 354 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:26.769685 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c4041-7b91-424f-b68f-0750743edff6 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:26.770521 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-095ce8ac-aa2f-4621-8133-8007a1b08e44 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:26.808860 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-941c4041-7b91-424f-b68f-0750743edff6 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:26.827392 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b270ccc-6d47-4617-82c6-855508e75fca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:26.828344 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b270ccc-6d47-4617-82c6-855508e75fca admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:26.828974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b270ccc-6d47-4617-82c6-855508e75fca admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:26.829271 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b270ccc-6d47-4617-82c6-855508e75fca admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:26.830491 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b270ccc-6d47-4617-82c6-855508e75fca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:26.836254 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b270ccc-6d47-4617-82c6-855508e75fca admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:26.921702 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-095ce8ac-aa2f-4621-8133-8007a1b08e44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2895da85d9d43c08bd41d2a729eb002'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:26.923333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-095ce8ac-aa2f-4621-8133-8007a1b08e44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2895da85d9d43c08bd41d2a729eb002'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:26.931308 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:26.945654 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 895/1789] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:26 2022] POST /identity/v3/projects => generated 379 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:26.949153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b270ccc-6d47-4617-82c6-855508e75fca admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:26.962577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-095ce8ac-aa2f-4621-8133-8007a1b08e44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2895da85d9d43c08bd41d2a729eb002'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:26.979248 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-095ce8ac-aa2f-4621-8133-8007a1b08e44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2895da85d9d43c08bd41d2a729eb002'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:26.979926 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-095ce8ac-aa2f-4621-8133-8007a1b08e44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2895da85d9d43c08bd41d2a729eb002'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:27.009534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 895/1790] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:26 2022] DELETE /identity/v3/users/f2895da85d9d43c08bd41d2a729eb002 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:27.010871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ce8ac-aa2f-4621-8133-8007a1b08e44 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:27.013534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-095ce8ac-aa2f-4621-8133-8007a1b08e44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:27.090424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b270ccc-6d47-4617-82c6-855508e75fca admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:27.150642 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-095ce8ac-aa2f-4621-8133-8007a1b08e44 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:27.161044 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419fa67-a489-4f27-9b26-9ba1586a2b18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:27.162271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419fa67-a489-4f27-9b26-9ba1586a2b18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:27.162462 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419fa67-a489-4f27-9b26-9ba1586a2b18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:27.162750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1419fa67-a489-4f27-9b26-9ba1586a2b18 admin admin] PATH_INFO: `/v3/projects/762c34908bf744d385b516f5b8d87b0f` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:27.166618 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419fa67-a489-4f27-9b26-9ba1586a2b18 admin admin] RBAC: Authorizing `identity:delete_project(project_id=762c34908bf744d385b516f5b8d87b0f)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:27.169849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1419fa67-a489-4f27-9b26-9ba1586a2b18 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:27.217809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:27.222111 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ef6f7-d60e-4324-9370-e742765af3f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:27.222823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ef6f7-d60e-4324-9370-e742765af3f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:27.223114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ef6f7-d60e-4324-9370-e742765af3f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:27.223412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ef6f7-d60e-4324-9370-e742765af3f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:27.225263 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ef6f7-d60e-4324-9370-e742765af3f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:27.226593 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-781ef6f7-d60e-4324-9370-e742765af3f8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:27.270187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 896/1791] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:26 2022] POST /identity/v3/users => generated 290 bytes in 321 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:27.271379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1419fa67-a489-4f27-9b26-9ba1586a2b18 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '762c34908bf744d385b516f5b8d87b0f', 'request_id': 'req-1419fa67-a489-4f27-9b26-9ba1586a2b18'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:27.274393 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ef6f7-d60e-4324-9370-e742765af3f8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:27.276178 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-781ef6f7-d60e-4324-9370-e742765af3f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:27.282028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1419fa67-a489-4f27-9b26-9ba1586a2b18 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '762c34908bf744d385b516f5b8d87b0f', 'request_id': 'req-1419fa67-a489-4f27-9b26-9ba1586a2b18'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:27.305656 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ef6f7-d60e-4324-9370-e742765af3f8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:27.312712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1419fa67-a489-4f27-9b26-9ba1586a2b18 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '762c34908bf744d385b516f5b8d87b0f', 'request_id': 'req-1419fa67-a489-4f27-9b26-9ba1586a2b18'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:27.336667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1419fa67-a489-4f27-9b26-9ba1586a2b18 admin admin] The token cache is being invalidate because project 762c34908bf744d385b516f5b8d87b0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:27.337237 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1419fa67-a489-4f27-9b26-9ba1586a2b18 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:27.337919 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1419fa67-a489-4f27-9b26-9ba1586a2b18 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:27.345797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 896/1792] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:27 2022] DELETE /identity/v3/projects/762c34908bf744d385b516f5b8d87b0f => generated 0 bytes in 334 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:27.348207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419fa67-a489-4f27-9b26-9ba1586a2b18 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:27.350434 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1419fa67-a489-4f27-9b26-9ba1586a2b18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:27.453329 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79d209-0a03-4cf7-8f3e-f0f79ae9b521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:27.453761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79d209-0a03-4cf7-8f3e-f0f79ae9b521 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:27.454482 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79d209-0a03-4cf7-8f3e-f0f79ae9b521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:27.455239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b79d209-0a03-4cf7-8f3e-f0f79ae9b521 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:27.456661 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79d209-0a03-4cf7-8f3e-f0f79ae9b521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:27.461705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b79d209-0a03-4cf7-8f3e-f0f79ae9b521 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:27.526610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 897/1793] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:27.534214 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79d209-0a03-4cf7-8f3e-f0f79ae9b521 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:27.561616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6b79d209-0a03-4cf7-8f3e-f0f79ae9b521 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:27.594371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1419fa67-a489-4f27-9b26-9ba1586a2b18 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:27.597235 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b79d209-0a03-4cf7-8f3e-f0f79ae9b521 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:27.615747 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f43ee9-7019-4d7d-a79e-961602eace90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:27.619059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f43ee9-7019-4d7d-a79e-961602eace90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:27.619991 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f43ee9-7019-4d7d-a79e-961602eace90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:27.620529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95836f64-9ff5-4f92-be54-717e8a4131df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:27.622278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f43ee9-7019-4d7d-a79e-961602eace90 admin admin] PATH_INFO: `/v3/users/7ca575d2be31474091996544b5498889` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:27.623803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95836f64-9ff5-4f92-be54-717e8a4131df admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:27.624631 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95836f64-9ff5-4f92-be54-717e8a4131df admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:27.624930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95836f64-9ff5-4f92-be54-717e8a4131df admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:27.625762 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f43ee9-7019-4d7d-a79e-961602eace90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ca575d2be31474091996544b5498889)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:27.628584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95836f64-9ff5-4f92-be54-717e8a4131df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:27.636452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14f43ee9-7019-4d7d-a79e-961602eace90 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:27.638445 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95836f64-9ff5-4f92-be54-717e8a4131df admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:27.710762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 898/1794] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:27 2022] GET /identity/v3/roles => generated 1420 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:27.712735 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95836f64-9ff5-4f92-be54-717e8a4131df admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:27.735698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95836f64-9ff5-4f92-be54-717e8a4131df admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:27.745739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16688c4-ae41-4873-8faa-4b6942da2048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:27.747061 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16688c4-ae41-4873-8faa-4b6942da2048 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:27.748872 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16688c4-ae41-4873-8faa-4b6942da2048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:27.749338 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16688c4-ae41-4873-8faa-4b6942da2048 admin admin] PATH_INFO: `/v3/projects/18ac1ec9ae154562b3c967a35b600c57/users/f7da528ca9ef48ebbd69fc684e289a04/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:27.792536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16688c4-ae41-4873-8faa-4b6942da2048 admin admin] RBAC: Authorizing `identity:create_grant(project_id=18ac1ec9ae154562b3c967a35b600c57, user_id=f7da528ca9ef48ebbd69fc684e289a04, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:27.800296 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-14f43ee9-7019-4d7d-a79e-961602eace90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ca575d2be31474091996544b5498889'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:27.801018 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-14f43ee9-7019-4d7d-a79e-961602eace90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ca575d2be31474091996544b5498889'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:27.802213 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16688c4-ae41-4873-8faa-4b6942da2048 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:27.825227 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:27.834277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-14f43ee9-7019-4d7d-a79e-961602eace90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ca575d2be31474091996544b5498889'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:27.839541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 899/1795] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:27 2022] PUT /identity/v3/projects/18ac1ec9ae154562b3c967a35b600c57/users/f7da528ca9ef48ebbd69fc684e289a04/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:27.850025 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784d0c8-680e-44ae-b11c-082ec4eee639 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:27.850469 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784d0c8-680e-44ae-b11c-082ec4eee639 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:27.850869 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6784d0c8-680e-44ae-b11c-082ec4eee639 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:27.860106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-14f43ee9-7019-4d7d-a79e-961602eace90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ca575d2be31474091996544b5498889'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:27.860756 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-14f43ee9-7019-4d7d-a79e-961602eace90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ca575d2be31474091996544b5498889'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:27.891362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6784d0c8-680e-44ae-b11c-082ec4eee639 None None] MFA Rules not processed for user `f7da528ca9ef48ebbd69fc684e289a04`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:27.904662 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6784d0c8-680e-44ae-b11c-082ec4eee639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:27.908815 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 897/1796] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:27 2022] DELETE /identity/v3/users/7ca575d2be31474091996544b5498889 => generated 0 bytes in 562 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:28.016480 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 900/1797] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:28.021365 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f43ee9-7019-4d7d-a79e-961602eace90 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:28.023160 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-14f43ee9-7019-4d7d-a79e-961602eace90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:28.074164 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:28.119151 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784d0c8-680e-44ae-b11c-082ec4eee639 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:28.121296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6784d0c8-680e-44ae-b11c-082ec4eee639 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:28.130687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f43ee9-7019-4d7d-a79e-961602eace90 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:28.145791 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e8df0-5035-4e95-8aad-5dac7c795758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:28.146610 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e8df0-5035-4e95-8aad-5dac7c795758 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:28.146890 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e8df0-5035-4e95-8aad-5dac7c795758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:28.147139 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-078e8df0-5035-4e95-8aad-5dac7c795758 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:28.153704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e8df0-5035-4e95-8aad-5dac7c795758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:28.156273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-078e8df0-5035-4e95-8aad-5dac7c795758 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:28.210961 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 898/1798] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:28 2022] POST /identity/v3/projects => generated 381 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:28.214885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e8df0-5035-4e95-8aad-5dac7c795758 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:28.282738 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6784d0c8-680e-44ae-b11c-082ec4eee639 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:28.324258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-078e8df0-5035-4e95-8aad-5dac7c795758 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:28.329894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2919e-ccd8-49a2-b16f-9efa6866e997 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:28.330786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2919e-ccd8-49a2-b16f-9efa6866e997 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:28.331477 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2919e-ccd8-49a2-b16f-9efa6866e997 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:28.331477 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f2919e-ccd8-49a2-b16f-9efa6866e997 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:28.332258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2919e-ccd8-49a2-b16f-9efa6866e997 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:28.334374 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f2919e-ccd8-49a2-b16f-9efa6866e997 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:28.362400 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 899/1799] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:28 2022] POST /identity/v3/users => generated 297 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:28.363352 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2919e-ccd8-49a2-b16f-9efa6866e997 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:28.379267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f2919e-ccd8-49a2-b16f-9efa6866e997 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:28.385009 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3ff1f-cf1f-4da7-a01e-fb4854f92d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:28.385709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3ff1f-cf1f-4da7-a01e-fb4854f92d4c admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:28.385998 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3ff1f-cf1f-4da7-a01e-fb4854f92d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:28.386282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a3ff1f-cf1f-4da7-a01e-fb4854f92d4c admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:28.387056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3ff1f-cf1f-4da7-a01e-fb4854f92d4c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:28.388738 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a3ff1f-cf1f-4da7-a01e-fb4854f92d4c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:28.406150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 900/1800] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:28.408239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a688e0c-2fe1-461b-a761-88428f72b0de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:28.409418 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a688e0c-2fe1-461b-a761-88428f72b0de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:28.409694 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a688e0c-2fe1-461b-a761-88428f72b0de admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:28.414844 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3ff1f-cf1f-4da7-a01e-fb4854f92d4c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:28.415056 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a688e0c-2fe1-461b-a761-88428f72b0de admin admin] PATH_INFO: `/v3/projects/db3bf79857a74695971d4f1f5e92ef60` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:28.420108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a688e0c-2fe1-461b-a761-88428f72b0de admin admin] RBAC: Authorizing `identity:delete_project(project_id=db3bf79857a74695971d4f1f5e92ef60)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:28.434276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a3ff1f-cf1f-4da7-a01e-fb4854f92d4c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:28.450176 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a688e0c-2fe1-461b-a761-88428f72b0de admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:28.451043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd645a2-6be7-45fe-a407-7ce1f1c8386e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:28.452010 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd645a2-6be7-45fe-a407-7ce1f1c8386e admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:28.452448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd645a2-6be7-45fe-a407-7ce1f1c8386e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:28.452751 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd645a2-6be7-45fe-a407-7ce1f1c8386e admin admin] PATH_INFO: `/v3/projects/797538c79be144b690f198aa4b0a6e85/users/3b30dd300360442c8eafe5df4ef6104a/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:28.486262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd645a2-6be7-45fe-a407-7ce1f1c8386e admin admin] RBAC: Authorizing `identity:create_grant(project_id=797538c79be144b690f198aa4b0a6e85, user_id=3b30dd300360442c8eafe5df4ef6104a, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:28.489054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edd645a2-6be7-45fe-a407-7ce1f1c8386e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:28.523719 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 901/1801] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:28 2022] PUT /identity/v3/projects/797538c79be144b690f198aa4b0a6e85/users/3b30dd300360442c8eafe5df4ef6104a/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:28.528726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd645a2-6be7-45fe-a407-7ce1f1c8386e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:28.590478 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:28.614183 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5a688e0c-2fe1-461b-a761-88428f72b0de admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db3bf79857a74695971d4f1f5e92ef60', 'request_id': 'req-5a688e0c-2fe1-461b-a761-88428f72b0de'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:28.639855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5a688e0c-2fe1-461b-a761-88428f72b0de admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db3bf79857a74695971d4f1f5e92ef60', 'request_id': 'req-5a688e0c-2fe1-461b-a761-88428f72b0de'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:28.660826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5a688e0c-2fe1-461b-a761-88428f72b0de admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db3bf79857a74695971d4f1f5e92ef60', 'request_id': 'req-5a688e0c-2fe1-461b-a761-88428f72b0de'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:28.664214 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd645a2-6be7-45fe-a407-7ce1f1c8386e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:28.677097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:28.687723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5a688e0c-2fe1-461b-a761-88428f72b0de admin admin] The token cache is being invalidate because project db3bf79857a74695971d4f1f5e92ef60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:28.688162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5a688e0c-2fe1-461b-a761-88428f72b0de admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:28.690579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5a688e0c-2fe1-461b-a761-88428f72b0de admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:28.692910 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 901/1802] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:28 2022] DELETE /identity/v3/projects/db3bf79857a74695971d4f1f5e92ef60 => generated 0 bytes in 575 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:28.693799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a688e0c-2fe1-461b-a761-88428f72b0de admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:28.699944 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5a688e0c-2fe1-461b-a761-88428f72b0de admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:28.811356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcac14a-4b83-4fa2-86bf-ddf785e20fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:28.812831 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcac14a-4b83-4fa2-86bf-ddf785e20fae admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:28.814563 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcac14a-4b83-4fa2-86bf-ddf785e20fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:28.815029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcac14a-4b83-4fa2-86bf-ddf785e20fae admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:28.817284 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcac14a-4b83-4fa2-86bf-ddf785e20fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:28.819496 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcac14a-4b83-4fa2-86bf-ddf785e20fae admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:28.836903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a688e0c-2fe1-461b-a761-88428f72b0de admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:28.843617 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13620fb1-41f5-428f-bdbe-fd55e76fc5e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:28.844903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13620fb1-41f5-428f-bdbe-fd55e76fc5e1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:28.845434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13620fb1-41f5-428f-bdbe-fd55e76fc5e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:28.846024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13620fb1-41f5-428f-bdbe-fd55e76fc5e1 admin admin] PATH_INFO: `/v3/projects/0ba8ffbaba534893bede4661d877beda` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:28.847517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13620fb1-41f5-428f-bdbe-fd55e76fc5e1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0ba8ffbaba534893bede4661d877beda)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:28.849855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13620fb1-41f5-428f-bdbe-fd55e76fc5e1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:28.857410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 902/1803] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:28 2022] GET /identity/v3/roles => generated 1420 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:28.857410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f67e86-9459-4ea3-be50-9adf90ac5ecb None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:28.857410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f67e86-9459-4ea3-be50-9adf90ac5ecb None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:28.857410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f67e86-9459-4ea3-be50-9adf90ac5ecb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:28.897448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-18f67e86-9459-4ea3-be50-9adf90ac5ecb None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:28.902690 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-18f67e86-9459-4ea3-be50-9adf90ac5ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:28.920935 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 903/1804] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 66 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:28.921889 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f67e86-9459-4ea3-be50-9adf90ac5ecb None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:28.928652 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-18f67e86-9459-4ea3-be50-9adf90ac5ecb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:28.945402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-13620fb1-41f5-428f-bdbe-fd55e76fc5e1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0ba8ffbaba534893bede4661d877beda', 'request_id': 'req-13620fb1-41f5-428f-bdbe-fd55e76fc5e1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:28.966638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f67e86-9459-4ea3-be50-9adf90ac5ecb None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:28.974300 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-13620fb1-41f5-428f-bdbe-fd55e76fc5e1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0ba8ffbaba534893bede4661d877beda', 'request_id': 'req-13620fb1-41f5-428f-bdbe-fd55e76fc5e1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:28.982659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-13620fb1-41f5-428f-bdbe-fd55e76fc5e1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0ba8ffbaba534893bede4661d877beda', 'request_id': 'req-13620fb1-41f5-428f-bdbe-fd55e76fc5e1'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:29.006398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-13620fb1-41f5-428f-bdbe-fd55e76fc5e1 admin admin] The token cache is being invalidate because project 0ba8ffbaba534893bede4661d877beda was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:29.006760 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-13620fb1-41f5-428f-bdbe-fd55e76fc5e1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:29.009080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-13620fb1-41f5-428f-bdbe-fd55e76fc5e1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:29.015599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 902/1805] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:28 2022] DELETE /identity/v3/projects/0ba8ffbaba534893bede4661d877beda => generated 0 bytes in 319 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:29.015599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af623f0-884b-4813-8483-bed45b502832 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:29.015599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af623f0-884b-4813-8483-bed45b502832 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:29.015599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8af623f0-884b-4813-8483-bed45b502832 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:29.044411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8af623f0-884b-4813-8483-bed45b502832 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:29.046017 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8af623f0-884b-4813-8483-bed45b502832 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:29.062812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c1aef-733c-4924-a55c-0ec6299149f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:29.063952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c1aef-733c-4924-a55c-0ec6299149f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:29.065492 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c1aef-733c-4924-a55c-0ec6299149f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:29.065956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567c1aef-733c-4924-a55c-0ec6299149f7 admin admin] PATH_INFO: `/v3/projects/797538c79be144b690f198aa4b0a6e85/users/3b30dd300360442c8eafe5df4ef6104a/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:29.068893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c1aef-733c-4924-a55c-0ec6299149f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=797538c79be144b690f198aa4b0a6e85, user_id=3b30dd300360442c8eafe5df4ef6104a, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:29.072474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567c1aef-733c-4924-a55c-0ec6299149f7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:29.088702 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 904/1806] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:28 2022] PUT /identity/v3/projects/797538c79be144b690f198aa4b0a6e85/users/3b30dd300360442c8eafe5df4ef6104a/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:29.089964 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c1aef-733c-4924-a55c-0ec6299149f7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:29.091941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-567c1aef-733c-4924-a55c-0ec6299149f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:29.143002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:29.178085 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 903/1807] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:29.179802 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b943e-c2e5-4b98-b82d-68badc253a08 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:29.180356 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b943e-c2e5-4b98-b82d-68badc253a08 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:29.183897 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-260b943e-c2e5-4b98-b82d-68badc253a08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:29.213843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567c1aef-733c-4924-a55c-0ec6299149f7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:29.227923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8e9d1-ae2b-41a1-892c-959b53ccbca0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:29.230221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8e9d1-ae2b-41a1-892c-959b53ccbca0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:29.232017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8e9d1-ae2b-41a1-892c-959b53ccbca0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:29.232017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd8e9d1-ae2b-41a1-892c-959b53ccbca0 admin admin] PATH_INFO: `/v3/users/91105958bb0242838cdb2f0e213c674e` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:29.239675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8e9d1-ae2b-41a1-892c-959b53ccbca0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91105958bb0242838cdb2f0e213c674e)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:29.243712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd8e9d1-ae2b-41a1-892c-959b53ccbca0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:29.248924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-260b943e-c2e5-4b98-b82d-68badc253a08 None None] MFA Rules not processed for user `3b30dd300360442c8eafe5df4ef6104a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:29.251012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-260b943e-c2e5-4b98-b82d-68badc253a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:29.352990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=94,keystone:INSERT=1,keystone:DELETE=5 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:29.368248 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6fd8e9d1-ae2b-41a1-892c-959b53ccbca0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91105958bb0242838cdb2f0e213c674e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:29.372064 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6fd8e9d1-ae2b-41a1-892c-959b53ccbca0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91105958bb0242838cdb2f0e213c674e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:29.393830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6fd8e9d1-ae2b-41a1-892c-959b53ccbca0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91105958bb0242838cdb2f0e213c674e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:29.407312 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6fd8e9d1-ae2b-41a1-892c-959b53ccbca0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91105958bb0242838cdb2f0e213c674e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:29.408092 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6fd8e9d1-ae2b-41a1-892c-959b53ccbca0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91105958bb0242838cdb2f0e213c674e'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:29.437112 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 905/1808] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:29 2022] DELETE /identity/v3/users/91105958bb0242838cdb2f0e213c674e => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:29.437112 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8e9d1-ae2b-41a1-892c-959b53ccbca0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:29.440631 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6fd8e9d1-ae2b-41a1-892c-959b53ccbca0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:29.455869 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 904/1809] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:29 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 276 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:29.475517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b943e-c2e5-4b98-b82d-68badc253a08 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:29.477745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-260b943e-c2e5-4b98-b82d-68badc253a08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:29.613661 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-260b943e-c2e5-4b98-b82d-68badc253a08 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:29.638817 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd8e9d1-ae2b-41a1-892c-959b53ccbca0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:29.643978 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039f057-c339-447a-ace0-e9587c17beb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:29.644715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039f057-c339-447a-ace0-e9587c17beb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:29.644972 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039f057-c339-447a-ace0-e9587c17beb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:29.645263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5039f057-c339-447a-ace0-e9587c17beb8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:29.647016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039f057-c339-447a-ace0-e9587c17beb8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:29.647424 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039f057-c339-447a-ace0-e9587c17beb8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:29.649498 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5039f057-c339-447a-ace0-e9587c17beb8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:29.671379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 906/1810] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:29.672352 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039f057-c339-447a-ace0-e9587c17beb8 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:29.683142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5039f057-c339-447a-ace0-e9587c17beb8 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:29.688106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff1735-4323-4b6e-a807-72c155528d62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:29.688803 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff1735-4323-4b6e-a807-72c155528d62 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:29.689118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff1735-4323-4b6e-a807-72c155528d62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:29.689386 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ff1735-4323-4b6e-a807-72c155528d62 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:29.690148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff1735-4323-4b6e-a807-72c155528d62 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:29.692237 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0ff1735-4323-4b6e-a807-72c155528d62 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:29.700748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8928a60-1d16-4049-9e43-4b069f110689 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:29.702230 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8928a60-1d16-4049-9e43-4b069f110689 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:29.702723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8928a60-1d16-4049-9e43-4b069f110689 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:29.703001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8928a60-1d16-4049-9e43-4b069f110689 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:29.751466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 907/1811] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:29 2022] POST /identity/v3/projects => generated 401 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:29.753144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff1735-4323-4b6e-a807-72c155528d62 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:29.824704 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:29.851395 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8928a60-1d16-4049-9e43-4b069f110689 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:29.874397 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8928a60-1d16-4049-9e43-4b069f110689 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:29.886801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ff1735-4323-4b6e-a807-72c155528d62 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:29.896074 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b0311-390b-46a8-8551-78bc848ddf72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:29.896074 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b0311-390b-46a8-8551-78bc848ddf72 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:29.896074 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b0311-390b-46a8-8551-78bc848ddf72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:29.896074 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-432b0311-390b-46a8-8551-78bc848ddf72 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:29.896391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b0311-390b-46a8-8551-78bc848ddf72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:29.898823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-432b0311-390b-46a8-8551-78bc848ddf72 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:29.911023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 905/1812] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:29 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:29.925390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8928a60-1d16-4049-9e43-4b069f110689 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:29.928786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b8928a60-1d16-4049-9e43-4b069f110689 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:29.953004 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 908/1813] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:29 2022] POST /identity/v3/users => generated 301 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:29.957732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b0311-390b-46a8-8551-78bc848ddf72 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:29.965182 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8928a60-1d16-4049-9e43-4b069f110689 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:29.971256 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0864158-674f-40fd-94c0-1d7241bd47ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:29.972539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0864158-674f-40fd-94c0-1d7241bd47ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:29.974894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0864158-674f-40fd-94c0-1d7241bd47ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:29.975376 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0864158-674f-40fd-94c0-1d7241bd47ba admin admin] PATH_INFO: `/v3/projects/4ba513ec36cb4a43a511c08727c83f83` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:29.979158 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-432b0311-390b-46a8-8551-78bc848ddf72 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:29.980096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0864158-674f-40fd-94c0-1d7241bd47ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ba513ec36cb4a43a511c08727c83f83)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:29.986448 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0864158-674f-40fd-94c0-1d7241bd47ba admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:30.002536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53f143-0117-4850-9c9e-363f9890138d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:30.003246 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53f143-0117-4850-9c9e-363f9890138d admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:30.003539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53f143-0117-4850-9c9e-363f9890138d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:30.003856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad53f143-0117-4850-9c9e-363f9890138d admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:30.004608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53f143-0117-4850-9c9e-363f9890138d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:30.007561 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad53f143-0117-4850-9c9e-363f9890138d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:30.034522 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 909/1814] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:29 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:30.040576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53f143-0117-4850-9c9e-363f9890138d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:30.056473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad53f143-0117-4850-9c9e-363f9890138d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:30.071169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:DELETE=5,keystone:SELECT=90,keystone:INSERT=5 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:30.080312 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e0864158-674f-40fd-94c0-1d7241bd47ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ba513ec36cb4a43a511c08727c83f83', 'request_id': 'req-e0864158-674f-40fd-94c0-1d7241bd47ba'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:30.096770 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e0864158-674f-40fd-94c0-1d7241bd47ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ba513ec36cb4a43a511c08727c83f83', 'request_id': 'req-e0864158-674f-40fd-94c0-1d7241bd47ba'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:30.124569 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e0864158-674f-40fd-94c0-1d7241bd47ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ba513ec36cb4a43a511c08727c83f83', 'request_id': 'req-e0864158-674f-40fd-94c0-1d7241bd47ba'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:30.152801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e0864158-674f-40fd-94c0-1d7241bd47ba admin admin] The token cache is being invalidate because project 4ba513ec36cb4a43a511c08727c83f83 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:30.153365 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e0864158-674f-40fd-94c0-1d7241bd47ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:30.154521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e0864158-674f-40fd-94c0-1d7241bd47ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:30.157533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 906/1815] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:29 2022] DELETE /identity/v3/projects/4ba513ec36cb4a43a511c08727c83f83 => generated 0 bytes in 233 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:30.163754 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1efcd-a4f5-40d5-9213-a7b791a65ab2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:30.164138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1efcd-a4f5-40d5-9213-a7b791a65ab2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:30.164855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e1efcd-a4f5-40d5-9213-a7b791a65ab2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:30.178192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db36eeb-5ba8-4c40-958f-a2e0aa0e2c14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:30.179979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db36eeb-5ba8-4c40-958f-a2e0aa0e2c14 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:30.180270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db36eeb-5ba8-4c40-958f-a2e0aa0e2c14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:30.180618 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db36eeb-5ba8-4c40-958f-a2e0aa0e2c14 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:30.181509 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db36eeb-5ba8-4c40-958f-a2e0aa0e2c14 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:30.183340 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db36eeb-5ba8-4c40-958f-a2e0aa0e2c14 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:30.183447 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-33e1efcd-a4f5-40d5-9213-a7b791a65ab2 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:30.184900 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-33e1efcd-a4f5-40d5-9213-a7b791a65ab2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:30.194829 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 910/1816] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:30.195858 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 907/1817] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:08:30.202837 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db36eeb-5ba8-4c40-958f-a2e0aa0e2c14 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:30.204315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba1161-e2f8-434b-a43d-ef22c56018f9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:30.204610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba1161-e2f8-434b-a43d-ef22c56018f9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:30.204707 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2db36eeb-5ba8-4c40-958f-a2e0aa0e2c14 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:30.205533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86ba1161-e2f8-434b-a43d-ef22c56018f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:30.218695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db36eeb-5ba8-4c40-958f-a2e0aa0e2c14 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:30.223976 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751998de-3e65-46d9-aaa9-d980131ef366 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:30.225017 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-86ba1161-e2f8-434b-a43d-ef22c56018f9 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:30.225135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751998de-3e65-46d9-aaa9-d980131ef366 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:30.225468 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751998de-3e65-46d9-aaa9-d980131ef366 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:30.225831 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751998de-3e65-46d9-aaa9-d980131ef366 admin admin] PATH_INFO: `/v3/projects/e7233dd5734d46e3b064c34fb4f1e244/users/df663d871de84844814a11a1ffdbca10/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:30.226328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-86ba1161-e2f8-434b-a43d-ef22c56018f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:30.239044 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 908/1818] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:30.244049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751998de-3e65-46d9-aaa9-d980131ef366 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7233dd5734d46e3b064c34fb4f1e244, user_id=df663d871de84844814a11a1ffdbca10, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:30.244936 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba1161-e2f8-434b-a43d-ef22c56018f9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:30.246957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751998de-3e65-46d9-aaa9-d980131ef366 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:30.258130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86ba1161-e2f8-434b-a43d-ef22c56018f9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:30.270808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 911/1819] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:30 2022] PUT /identity/v3/projects/e7233dd5734d46e3b064c34fb4f1e244/users/df663d871de84844814a11a1ffdbca10/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:30.281253 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d9f51-9d87-4707-8783-186aecb0c4df None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:30.281572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d9f51-9d87-4707-8783-186aecb0c4df None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:30.281828 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-308d9f51-9d87-4707-8783-186aecb0c4df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:30.329670 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-308d9f51-9d87-4707-8783-186aecb0c4df None None] MFA Rules not processed for user `df663d871de84844814a11a1ffdbca10`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:30.331109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-308d9f51-9d87-4707-8783-186aecb0c4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:30.341104 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca83f9-15c6-487f-bde6-c0945d0ec94c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:30.343542 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca83f9-15c6-487f-bde6-c0945d0ec94c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:30.343860 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca83f9-15c6-487f-bde6-c0945d0ec94c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:30.345393 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdca83f9-15c6-487f-bde6-c0945d0ec94c admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:30.347503 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca83f9-15c6-487f-bde6-c0945d0ec94c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:30.347993 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca83f9-15c6-487f-bde6-c0945d0ec94c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:30.352195 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdca83f9-15c6-487f-bde6-c0945d0ec94c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:30.395485 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 909/1820] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:30.400099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca83f9-15c6-487f-bde6-c0945d0ec94c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:30.435223 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdca83f9-15c6-487f-bde6-c0945d0ec94c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:30.457597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3442f81-a361-4117-9ebf-27c8bc571963 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:30.458653 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3442f81-a361-4117-9ebf-27c8bc571963 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:30.458942 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3442f81-a361-4117-9ebf-27c8bc571963 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:30.459221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3442f81-a361-4117-9ebf-27c8bc571963 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:30.461747 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3442f81-a361-4117-9ebf-27c8bc571963 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:30.465438 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 912/1821] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:30 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 185 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:08:30.466726 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3442f81-a361-4117-9ebf-27c8bc571963 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:30.469465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d9f51-9d87-4707-8783-186aecb0c4df None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:30.471481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-308d9f51-9d87-4707-8783-186aecb0c4df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:30.535326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 910/1822] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:30 2022] POST /identity/v3/projects => generated 361 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:30.535326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3442f81-a361-4117-9ebf-27c8bc571963 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:30.598958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-308d9f51-9d87-4707-8783-186aecb0c4df None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:30.602539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:30.623749 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:30.666837 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3442f81-a361-4117-9ebf-27c8bc571963 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:30.674871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97863c66-f7d5-445a-b9ea-0249d750caec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:30.675981 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97863c66-f7d5-445a-b9ea-0249d750caec admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:30.676497 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97863c66-f7d5-445a-b9ea-0249d750caec admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:30.677014 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97863c66-f7d5-445a-b9ea-0249d750caec admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:30.680067 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97863c66-f7d5-445a-b9ea-0249d750caec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:30.684186 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97863c66-f7d5-445a-b9ea-0249d750caec admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:30.699421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5833f2-7e60-4129-8e4c-265a3b27b7ac tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b30dd300360442c8eafe5df4ef6104a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '797538c79be144b690f198aa4b0a6e85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:30.700149 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5833f2-7e60-4129-8e4c-265a3b27b7ac tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:30.700444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5833f2-7e60-4129-8e4c-265a3b27b7ac tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:30.700739 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad5833f2-7e60-4129-8e4c-265a3b27b7ac tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:30.703532 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5833f2-7e60-4129-8e4c-265a3b27b7ac tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:30.703532 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5833f2-7e60-4129-8e4c-265a3b27b7ac tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:30.704401 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad5833f2-7e60-4129-8e4c-265a3b27b7ac tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:30.721725 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 913/1823] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:30.721725 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5833f2-7e60-4129-8e4c-265a3b27b7ac tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:30.733816 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad5833f2-7e60-4129-8e4c-265a3b27b7ac tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:30.739254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f722c8a1-75f7-414b-a899-6d0541541836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:30.740172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722c8a1-75f7-414b-a899-6d0541541836 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:30.743590 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722c8a1-75f7-414b-a899-6d0541541836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:30.743590 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f722c8a1-75f7-414b-a899-6d0541541836 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:30.743590 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f722c8a1-75f7-414b-a899-6d0541541836 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:30.743888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f722c8a1-75f7-414b-a899-6d0541541836 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:30.747619 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 911/1824] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:30 2022] POST /identity/v3/projects => generated 401 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:30.748856 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97863c66-f7d5-445a-b9ea-0249d750caec admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:30.770503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 914/1825] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:30 2022] POST /identity/v3/users => generated 281 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:30.771534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f722c8a1-75f7-414b-a899-6d0541541836 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:30.783573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f722c8a1-75f7-414b-a899-6d0541541836 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:30.847576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97863c66-f7d5-445a-b9ea-0249d750caec admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:30.854991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c339621-218f-47b6-89c6-37fe0cc6c5b0 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b30dd300360442c8eafe5df4ef6104a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '797538c79be144b690f198aa4b0a6e85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:30.857397 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429d8d9-bb96-461c-865a-8816050c3658 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:30.858562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c339621-218f-47b6-89c6-37fe0cc6c5b0 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:30.858863 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c339621-218f-47b6-89c6-37fe0cc6c5b0 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:30.859177 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c339621-218f-47b6-89c6-37fe0cc6c5b0 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:30.860249 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429d8d9-bb96-461c-865a-8816050c3658 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:30.860540 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429d8d9-bb96-461c-865a-8816050c3658 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:30.861333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1429d8d9-bb96-461c-865a-8816050c3658 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:30.861733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c339621-218f-47b6-89c6-37fe0cc6c5b0 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:30.863763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c339621-218f-47b6-89c6-37fe0cc6c5b0 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:30.865565 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1429d8d9-bb96-461c-865a-8816050c3658 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:30.867693 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1429d8d9-bb96-461c-865a-8816050c3658 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:30.909767 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 915/1826] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:30 2022] POST /identity/v3/users => generated 307 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:30.909767 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429d8d9-bb96-461c-865a-8816050c3658 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:30.916713 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 912/1827] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:30 2022] POST /identity/v3/projects => generated 354 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:30.916713 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c339621-218f-47b6-89c6-37fe0cc6c5b0 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:30.931909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1429d8d9-bb96-461c-865a-8816050c3658 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:31.045644 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7967a26-a09c-4e08-a6a8-bb9332170596 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:31.046470 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7967a26-a09c-4e08-a6a8-bb9332170596 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:31.046835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7967a26-a09c-4e08-a6a8-bb9332170596 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:31.047191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7967a26-a09c-4e08-a6a8-bb9332170596 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:31.048052 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7967a26-a09c-4e08-a6a8-bb9332170596 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:31.053158 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7967a26-a09c-4e08-a6a8-bb9332170596 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:31.053526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c339621-218f-47b6-89c6-37fe0cc6c5b0 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:31.063024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d0bd1-3118-4148-8d5b-c4fc7559d834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:31.063773 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d0bd1-3118-4148-8d5b-c4fc7559d834 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:31.064179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d0bd1-3118-4148-8d5b-c4fc7559d834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:31.064734 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-405d0bd1-3118-4148-8d5b-c4fc7559d834 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:31.065495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d0bd1-3118-4148-8d5b-c4fc7559d834 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:31.067247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-405d0bd1-3118-4148-8d5b-c4fc7559d834 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:31.069835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 916/1828] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:31.070874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7967a26-a09c-4e08-a6a8-bb9332170596 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:31.095884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 913/1829] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:30 2022] GET /identity/v3/roles => generated 1420 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:31.097105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d0bd1-3118-4148-8d5b-c4fc7559d834 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:31.097414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:31.108789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-405d0bd1-3118-4148-8d5b-c4fc7559d834 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:31.114150 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e30326-e574-4a4a-a5c2-a4dd8d6121ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:31.114890 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e30326-e574-4a4a-a5c2-a4dd8d6121ae admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:31.115321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e30326-e574-4a4a-a5c2-a4dd8d6121ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:31.115615 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13e30326-e574-4a4a-a5c2-a4dd8d6121ae admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:31.116438 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e30326-e574-4a4a-a5c2-a4dd8d6121ae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:31.118150 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13e30326-e574-4a4a-a5c2-a4dd8d6121ae admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:31.129461 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 914/1830] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:31.130312 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e30326-e574-4a4a-a5c2-a4dd8d6121ae admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:31.142397 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13e30326-e574-4a4a-a5c2-a4dd8d6121ae admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:31.149209 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bab23-fa37-4c55-ae96-ca66537624b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:31.150199 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bab23-fa37-4c55-ae96-ca66537624b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:31.150493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bab23-fa37-4c55-ae96-ca66537624b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:31.150838 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-231bab23-fa37-4c55-ae96-ca66537624b4 admin admin] PATH_INFO: `/v3/projects/1a62b18e2924417d81f957f1aae67e88/users/68e3d7dde5ab4f6eb165b7f197100d74/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:31.170274 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bab23-fa37-4c55-ae96-ca66537624b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a62b18e2924417d81f957f1aae67e88, user_id=68e3d7dde5ab4f6eb165b7f197100d74, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:31.173119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-231bab23-fa37-4c55-ae96-ca66537624b4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:31.183723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7967a26-a09c-4e08-a6a8-bb9332170596 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:31.189387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86af99a-0012-4f38-ac84-8d51ba8129dc tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b30dd300360442c8eafe5df4ef6104a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '797538c79be144b690f198aa4b0a6e85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:31.190277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86af99a-0012-4f38-ac84-8d51ba8129dc tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:31.190812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86af99a-0012-4f38-ac84-8d51ba8129dc tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:31.191107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86af99a-0012-4f38-ac84-8d51ba8129dc tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:31.191761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:31.192013 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86af99a-0012-4f38-ac84-8d51ba8129dc tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:31.192345 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86af99a-0012-4f38-ac84-8d51ba8129dc tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:31.194443 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86af99a-0012-4f38-ac84-8d51ba8129dc tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:31.198422 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 915/1831] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:31 2022] PUT /identity/v3/projects/1a62b18e2924417d81f957f1aae67e88/users/68e3d7dde5ab4f6eb165b7f197100d74/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:31.203730 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bab23-fa37-4c55-ae96-ca66537624b4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:31.209345 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 917/1832] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:31.210510 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86af99a-0012-4f38-ac84-8d51ba8129dc tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:31.236957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86af99a-0012-4f38-ac84-8d51ba8129dc tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:31.341547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b5262-2a8d-41c9-bbe6-628589c28172 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:31.342101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b5262-2a8d-41c9-bbe6-628589c28172 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:31.342394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b5262-2a8d-41c9-bbe6-628589c28172 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:31.342684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46b5262-2a8d-41c9-bbe6-628589c28172 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:31.343433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b5262-2a8d-41c9-bbe6-628589c28172 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:31.346390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-231bab23-fa37-4c55-ae96-ca66537624b4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:31.348956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46b5262-2a8d-41c9-bbe6-628589c28172 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:31.358219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930010e-a109-421a-82ec-782edd93464a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:31.359149 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930010e-a109-421a-82ec-782edd93464a admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:31.359441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930010e-a109-421a-82ec-782edd93464a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:31.359913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d930010e-a109-421a-82ec-782edd93464a admin admin] PATH_INFO: `/v3/projects/47a0a382993e438a8ae1e67f7232290e/users/4a8a53e80d824213adaceb0bcf00669a/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:31.375547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 918/1833] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:31.378826 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b5262-2a8d-41c9-bbe6-628589c28172 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:31.405433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930010e-a109-421a-82ec-782edd93464a admin admin] RBAC: Authorizing `identity:create_grant(project_id=47a0a382993e438a8ae1e67f7232290e, user_id=4a8a53e80d824213adaceb0bcf00669a, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:31.409137 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d930010e-a109-421a-82ec-782edd93464a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:31.437208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 916/1834] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:31 2022] PUT /identity/v3/projects/47a0a382993e438a8ae1e67f7232290e/users/4a8a53e80d824213adaceb0bcf00669a/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:31.438172 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930010e-a109-421a-82ec-782edd93464a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:31.496816 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46b5262-2a8d-41c9-bbe6-628589c28172 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:31.564428 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d930010e-a109-421a-82ec-782edd93464a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:31.573913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f25cf0-3eb0-40fb-b8b9-884ed3b7db2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:31.574962 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f25cf0-3eb0-40fb-b8b9-884ed3b7db2b admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:31.575356 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f25cf0-3eb0-40fb-b8b9-884ed3b7db2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:31.575742 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f25cf0-3eb0-40fb-b8b9-884ed3b7db2b admin admin] PATH_INFO: `/v3/projects/1a62b18e2924417d81f957f1aae67e88/users/68e3d7dde5ab4f6eb165b7f197100d74/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:31.577911 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f25cf0-3eb0-40fb-b8b9-884ed3b7db2b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a62b18e2924417d81f957f1aae67e88, user_id=68e3d7dde5ab4f6eb165b7f197100d74, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:31.580772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f25cf0-3eb0-40fb-b8b9-884ed3b7db2b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:31.604326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 917/1835] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:31 2022] PUT /identity/v3/projects/1a62b18e2924417d81f957f1aae67e88/users/68e3d7dde5ab4f6eb165b7f197100d74/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:31.606463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566b92b-f6f5-4d6f-a67f-db64c9bc707d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:31.606790 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566b92b-f6f5-4d6f-a67f-db64c9bc707d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:31.607138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566b92b-f6f5-4d6f-a67f-db64c9bc707d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:31.630984 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:31.641147 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce13a1f2-d708-4814-a2dc-fa0fc5e66538 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '3b30dd300360442c8eafe5df4ef6104a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '797538c79be144b690f198aa4b0a6e85', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:31.642396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13a1f2-d708-4814-a2dc-fa0fc5e66538 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:31.642921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13a1f2-d708-4814-a2dc-fa0fc5e66538 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:31.643381 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce13a1f2-d708-4814-a2dc-fa0fc5e66538 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] PATH_INFO: `/v3/projects/50375c8f70c8453aa328b6e4ef87b00d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:31.644958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13a1f2-d708-4814-a2dc-fa0fc5e66538 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] RBAC: Authorizing `identity:delete_project(project_id=50375c8f70c8453aa328b6e4ef87b00d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:31.647140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce13a1f2-d708-4814-a2dc-fa0fc5e66538 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:31.657174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3566b92b-f6f5-4d6f-a67f-db64c9bc707d None None] MFA Rules not processed for user `4a8a53e80d824213adaceb0bcf00669a`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:31.662411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3566b92b-f6f5-4d6f-a67f-db64c9bc707d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:31.712083 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ce13a1f2-d708-4814-a2dc-fa0fc5e66538 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50375c8f70c8453aa328b6e4ef87b00d', 'request_id': 'req-ce13a1f2-d708-4814-a2dc-fa0fc5e66538'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:31.719306 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ce13a1f2-d708-4814-a2dc-fa0fc5e66538 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50375c8f70c8453aa328b6e4ef87b00d', 'request_id': 'req-ce13a1f2-d708-4814-a2dc-fa0fc5e66538'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:31.730992 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ce13a1f2-d708-4814-a2dc-fa0fc5e66538 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50375c8f70c8453aa328b6e4ef87b00d', 'request_id': 'req-ce13a1f2-d708-4814-a2dc-fa0fc5e66538'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:31.746410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ce13a1f2-d708-4814-a2dc-fa0fc5e66538 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] The token cache is being invalidate because project 50375c8f70c8453aa328b6e4ef87b00d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:31.746716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ce13a1f2-d708-4814-a2dc-fa0fc5e66538 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:31.749760 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ce13a1f2-d708-4814-a2dc-fa0fc5e66538 tempest-VolumeQuotaClassesTest-1015087258 tempest-VolumeQuotaClassesTest-1015087258-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:31.752488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 919/1836] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:31 2022] DELETE /identity/v3/projects/50375c8f70c8453aa328b6e4ef87b00d => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:31.758588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 918/1837] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:31 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:31.759483 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189916de-72b7-4be8-8744-50f4d9551fb8 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:31.759483 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189916de-72b7-4be8-8744-50f4d9551fb8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:31.759483 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-189916de-72b7-4be8-8744-50f4d9551fb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:31.836744 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566b92b-f6f5-4d6f-a67f-db64c9bc707d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:31.838764 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3566b92b-f6f5-4d6f-a67f-db64c9bc707d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:31.882133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:31.886802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-189916de-72b7-4be8-8744-50f4d9551fb8 None None] MFA Rules not processed for user `68e3d7dde5ab4f6eb165b7f197100d74`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:31.889918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-189916de-72b7-4be8-8744-50f4d9551fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:31.996351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566b92b-f6f5-4d6f-a67f-db64c9bc707d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:32.101331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 920/1838] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:31 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 348 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:32.137029 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ee48e-6680-4842-8fda-9f1c47643123 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:32.137717 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ee48e-6680-4842-8fda-9f1c47643123 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:32.138000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ee48e-6680-4842-8fda-9f1c47643123 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:32.138283 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122ee48e-6680-4842-8fda-9f1c47643123 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:32.143513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-122ee48e-6680-4842-8fda-9f1c47643123 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:32.164673 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189916de-72b7-4be8-8744-50f4d9551fb8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:32.166105 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-189916de-72b7-4be8-8744-50f4d9551fb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:32.182448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-189916de-72b7-4be8-8744-50f4d9551fb8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:32.201067 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90eacbb-6ec0-47a7-b504-9c06756f0b22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:32.201872 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90eacbb-6ec0-47a7-b504-9c06756f0b22 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:32.202097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90eacbb-6ec0-47a7-b504-9c06756f0b22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:32.202434 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90eacbb-6ec0-47a7-b504-9c06756f0b22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:32.217852 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90eacbb-6ec0-47a7-b504-9c06756f0b22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:32.219321 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90eacbb-6ec0-47a7-b504-9c06756f0b22 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:32.253168 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ee48e-6680-4842-8fda-9f1c47643123 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:32.254722 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122ee48e-6680-4842-8fda-9f1c47643123 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:32.260227 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 921/1839] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:32 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:08:32.267675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 919/1840] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:31 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 431 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:32.283740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90eacbb-6ec0-47a7-b504-9c06756f0b22 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:32.285364 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e90eacbb-6ec0-47a7-b504-9c06756f0b22 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:32.343926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ee48e-6680-4842-8fda-9f1c47643123 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:32.345500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-122ee48e-6680-4842-8fda-9f1c47643123 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:32.417429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:32.424504 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90eacbb-6ec0-47a7-b504-9c06756f0b22 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:32.431260 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0ab4f-91d8-4527-b0f0-13ca8e008b36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:32.432833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0ab4f-91d8-4527-b0f0-13ca8e008b36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:32.433313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0ab4f-91d8-4527-b0f0-13ca8e008b36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:32.433685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0ab4f-91d8-4527-b0f0-13ca8e008b36 admin admin] PATH_INFO: `/v3/users/4a8a53e80d824213adaceb0bcf00669a` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:32.436081 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0ab4f-91d8-4527-b0f0-13ca8e008b36 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a8a53e80d824213adaceb0bcf00669a)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:32.439340 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0ab4f-91d8-4527-b0f0-13ca8e008b36 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:32.447025 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122ee48e-6680-4842-8fda-9f1c47643123 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:32.453637 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3869f2a1-64d9-4503-9fab-9f47fb56b869 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:32.454574 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869f2a1-64d9-4503-9fab-9f47fb56b869 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:32.454854 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869f2a1-64d9-4503-9fab-9f47fb56b869 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:32.455621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3869f2a1-64d9-4503-9fab-9f47fb56b869 admin admin] PATH_INFO: `/v3/users/f7da528ca9ef48ebbd69fc684e289a04` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:32.459870 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869f2a1-64d9-4503-9fab-9f47fb56b869 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f7da528ca9ef48ebbd69fc684e289a04)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:32.462547 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3869f2a1-64d9-4503-9fab-9f47fb56b869 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:32.494584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:32.530368 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8de0ab4f-91d8-4527-b0f0-13ca8e008b36 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a8a53e80d824213adaceb0bcf00669a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:32.532123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8de0ab4f-91d8-4527-b0f0-13ca8e008b36 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a8a53e80d824213adaceb0bcf00669a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:32.541100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3869f2a1-64d9-4503-9fab-9f47fb56b869 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f7da528ca9ef48ebbd69fc684e289a04'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:32.549023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8de0ab4f-91d8-4527-b0f0-13ca8e008b36 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a8a53e80d824213adaceb0bcf00669a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:32.558794 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8de0ab4f-91d8-4527-b0f0-13ca8e008b36 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a8a53e80d824213adaceb0bcf00669a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:32.560589 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8de0ab4f-91d8-4527-b0f0-13ca8e008b36 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a8a53e80d824213adaceb0bcf00669a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:32.581248 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3869f2a1-64d9-4503-9fab-9f47fb56b869 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f7da528ca9ef48ebbd69fc684e289a04'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:32.595186 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3869f2a1-64d9-4503-9fab-9f47fb56b869 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f7da528ca9ef48ebbd69fc684e289a04'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:32.596041 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3869f2a1-64d9-4503-9fab-9f47fb56b869 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f7da528ca9ef48ebbd69fc684e289a04'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:32.608609 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3869f2a1-64d9-4503-9fab-9f47fb56b869 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f7da528ca9ef48ebbd69fc684e289a04'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:32.609048 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 922/1841] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:32 2022] DELETE /identity/v3/users/4a8a53e80d824213adaceb0bcf00669a => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:32.610273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0ab4f-91d8-4527-b0f0-13ca8e008b36 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:32.611895 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8de0ab4f-91d8-4527-b0f0-13ca8e008b36 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:32.613319 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 920/1842] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:32 2022] DELETE /identity/v3/users/f7da528ca9ef48ebbd69fc684e289a04 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:32.614914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218b703-9ac7-478a-878f-1c447defd3d6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:32.615219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218b703-9ac7-478a-878f-1c447defd3d6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:32.615777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2218b703-9ac7-478a-878f-1c447defd3d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:32.637295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2218b703-9ac7-478a-878f-1c447defd3d6 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:32.638817 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2218b703-9ac7-478a-878f-1c447defd3d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:32.758579 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0ab4f-91d8-4527-b0f0-13ca8e008b36 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:32.827934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 921/1843] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:32.836057 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467a18f-5150-420a-948e-ba285996e583 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:32.836373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467a18f-5150-420a-948e-ba285996e583 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:32.836661 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1467a18f-5150-420a-948e-ba285996e583 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:32.859495 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a39d99e-036a-4006-b45d-32e761a908e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:32.860394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39d99e-036a-4006-b45d-32e761a908e9 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:32.861990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39d99e-036a-4006-b45d-32e761a908e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:32.861990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a39d99e-036a-4006-b45d-32e761a908e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:32.869396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7a39d99e-036a-4006-b45d-32e761a908e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:32.878019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1467a18f-5150-420a-948e-ba285996e583 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:32.879251 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1467a18f-5150-420a-948e-ba285996e583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:32.893353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 922/1844] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:32.894384 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467a18f-5150-420a-948e-ba285996e583 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:32.896050 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1467a18f-5150-420a-948e-ba285996e583 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:32.948825 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:32.976802 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1467a18f-5150-420a-948e-ba285996e583 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:32.980696 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd8af1-3985-4899-b665-c4aa84e23162 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:32.981525 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd8af1-3985-4899-b665-c4aa84e23162 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:32.981924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd8af1-3985-4899-b665-c4aa84e23162 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:32.982254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a39d99e-036a-4006-b45d-32e761a908e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:32.982369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd8af1-3985-4899-b665-c4aa84e23162 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:32.983741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a39d99e-036a-4006-b45d-32e761a908e9 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:32.992820 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd8af1-3985-4899-b665-c4aa84e23162 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:32.994561 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cd8af1-3985-4899-b665-c4aa84e23162 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:33.006204 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 923/1845] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:33.007299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd8af1-3985-4899-b665-c4aa84e23162 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:33.015550 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 923/1846] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:32 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:33.018094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cd8af1-3985-4899-b665-c4aa84e23162 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:33.023954 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fad73-2292-4e51-9955-ab6227fbd72f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:33.024775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fad73-2292-4e51-9955-ab6227fbd72f service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:33.025151 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fad73-2292-4e51-9955-ab6227fbd72f service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:33.025544 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20fad73-2292-4e51-9955-ab6227fbd72f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:33.036657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fad73-2292-4e51-9955-ab6227fbd72f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:33.038352 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20fad73-2292-4e51-9955-ab6227fbd72f service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:33.050436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 924/1847] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:34.065811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a39d99e-036a-4006-b45d-32e761a908e9 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:34.067806 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7a39d99e-036a-4006-b45d-32e761a908e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:34.081952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a39d99e-036a-4006-b45d-32e761a908e9 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:34.090239 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e027cd-cf76-415d-8a67-75466a0488d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:34.091159 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e027cd-cf76-415d-8a67-75466a0488d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:34.091484 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e027cd-cf76-415d-8a67-75466a0488d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:34.091726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67e027cd-cf76-415d-8a67-75466a0488d2 admin admin] PATH_INFO: `/v3/projects/18ac1ec9ae154562b3c967a35b600c57` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:34.093890 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e027cd-cf76-415d-8a67-75466a0488d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18ac1ec9ae154562b3c967a35b600c57)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:34.095982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67e027cd-cf76-415d-8a67-75466a0488d2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:34.179698 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-67e027cd-cf76-415d-8a67-75466a0488d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18ac1ec9ae154562b3c967a35b600c57', 'request_id': 'req-67e027cd-cf76-415d-8a67-75466a0488d2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:34.187746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-67e027cd-cf76-415d-8a67-75466a0488d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18ac1ec9ae154562b3c967a35b600c57', 'request_id': 'req-67e027cd-cf76-415d-8a67-75466a0488d2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:34.207512 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-67e027cd-cf76-415d-8a67-75466a0488d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18ac1ec9ae154562b3c967a35b600c57', 'request_id': 'req-67e027cd-cf76-415d-8a67-75466a0488d2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:34.235353 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-67e027cd-cf76-415d-8a67-75466a0488d2 admin admin] The token cache is being invalidate because project 18ac1ec9ae154562b3c967a35b600c57 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:34.235353 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-67e027cd-cf76-415d-8a67-75466a0488d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:34.237268 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-67e027cd-cf76-415d-8a67-75466a0488d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:34.249470 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 924/1848] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:34 2022] DELETE /identity/v3/projects/18ac1ec9ae154562b3c967a35b600c57 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:34.250669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fad73-2292-4e51-9955-ab6227fbd72f service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:34.262816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20fad73-2292-4e51-9955-ab6227fbd72f service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:34.287204 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=8,keystone:SELECT=91,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:34.307168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e027cd-cf76-415d-8a67-75466a0488d2 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:34.313039 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-67e027cd-cf76-415d-8a67-75466a0488d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:34.379312 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ffa2966-29ab-403a-ac0b-b087ab818e6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:34.380432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa2966-29ab-403a-ac0b-b087ab818e6c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:34.380750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa2966-29ab-403a-ac0b-b087ab818e6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:34.381129 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ffa2966-29ab-403a-ac0b-b087ab818e6c admin admin] PATH_INFO: `/v3/users/3b30dd300360442c8eafe5df4ef6104a` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:34.388489 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa2966-29ab-403a-ac0b-b087ab818e6c admin admin] RBAC: Authorizing `identity:delete_user(user_id=3b30dd300360442c8eafe5df4ef6104a)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:34.390471 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ffa2966-29ab-403a-ac0b-b087ab818e6c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:34.465144 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6ffa2966-29ab-403a-ac0b-b087ab818e6c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3b30dd300360442c8eafe5df4ef6104a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:34.483410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67e027cd-cf76-415d-8a67-75466a0488d2 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:34.495112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6ffa2966-29ab-403a-ac0b-b087ab818e6c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3b30dd300360442c8eafe5df4ef6104a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:34.500244 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94217ae-59ba-4c27-8577-88887a89230b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:34.501335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94217ae-59ba-4c27-8577-88887a89230b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:34.501809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94217ae-59ba-4c27-8577-88887a89230b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:34.502222 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94217ae-59ba-4c27-8577-88887a89230b admin admin] PATH_INFO: `/v3/projects/47a0a382993e438a8ae1e67f7232290e` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:34.503844 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94217ae-59ba-4c27-8577-88887a89230b admin admin] RBAC: Authorizing `identity:delete_project(project_id=47a0a382993e438a8ae1e67f7232290e)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:34.506232 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94217ae-59ba-4c27-8577-88887a89230b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:34.516650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6ffa2966-29ab-403a-ac0b-b087ab818e6c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3b30dd300360442c8eafe5df4ef6104a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:34.517980 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6ffa2966-29ab-403a-ac0b-b087ab818e6c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3b30dd300360442c8eafe5df4ef6104a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:34.530179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6ffa2966-29ab-403a-ac0b-b087ab818e6c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3b30dd300360442c8eafe5df4ef6104a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:34.544350 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 925/1849] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:34 2022] DELETE /identity/v3/users/3b30dd300360442c8eafe5df4ef6104a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:34.605474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a94217ae-59ba-4c27-8577-88887a89230b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47a0a382993e438a8ae1e67f7232290e', 'request_id': 'req-a94217ae-59ba-4c27-8577-88887a89230b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:34.612532 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a94217ae-59ba-4c27-8577-88887a89230b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47a0a382993e438a8ae1e67f7232290e', 'request_id': 'req-a94217ae-59ba-4c27-8577-88887a89230b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:34.627408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a94217ae-59ba-4c27-8577-88887a89230b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47a0a382993e438a8ae1e67f7232290e', 'request_id': 'req-a94217ae-59ba-4c27-8577-88887a89230b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:34.663312 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:34.668215 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a94217ae-59ba-4c27-8577-88887a89230b admin admin] The token cache is being invalidate because project 47a0a382993e438a8ae1e67f7232290e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:34.668762 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a94217ae-59ba-4c27-8577-88887a89230b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:34.669471 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a94217ae-59ba-4c27-8577-88887a89230b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:34.671415 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 925/1850] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:34 2022] DELETE /identity/v3/projects/47a0a382993e438a8ae1e67f7232290e => generated 0 bytes in 365 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:34.678040 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66bb29-25f0-4c95-b757-b211c8f21b22 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:34.682156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66bb29-25f0-4c95-b757-b211c8f21b22 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:34.682582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db66bb29-25f0-4c95-b757-b211c8f21b22 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:34.737579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-db66bb29-25f0-4c95-b757-b211c8f21b22 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:34.739506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-db66bb29-25f0-4c95-b757-b211c8f21b22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:34.860240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 926/1851] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:34.868524 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b0e89-aad3-439a-b0df-9d77bcb0b694 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:34.868877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b0e89-aad3-439a-b0df-9d77bcb0b694 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:34.869154 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5b0e89-aad3-439a-b0df-9d77bcb0b694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:34.889676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-5c5b0e89-aad3-439a-b0df-9d77bcb0b694 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:34.891020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5c5b0e89-aad3-439a-b0df-9d77bcb0b694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:34.903180 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 926/1852] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:34.904921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66bb29-25f0-4c95-b757-b211c8f21b22 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:34.916230 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db66bb29-25f0-4c95-b757-b211c8f21b22 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:34.921162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25fbc6-488c-458a-b641-2240291aca05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:34.921841 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25fbc6-488c-458a-b641-2240291aca05 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:34.922127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25fbc6-488c-458a-b641-2240291aca05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:34.922438 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f25fbc6-488c-458a-b641-2240291aca05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:34.923209 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25fbc6-488c-458a-b641-2240291aca05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:34.923517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25fbc6-488c-458a-b641-2240291aca05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:34.925266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f25fbc6-488c-458a-b641-2240291aca05 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:34.939022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 927/1853] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:34.943700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b0e89-aad3-439a-b0df-9d77bcb0b694 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:34.954186 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5b0e89-aad3-439a-b0df-9d77bcb0b694 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:34.960859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25fbc6-488c-458a-b641-2240291aca05 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:34.963297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3f25fbc6-488c-458a-b641-2240291aca05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:34.964774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8348209-28de-48cb-80fb-de5494add839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:34.965895 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8348209-28de-48cb-80fb-de5494add839 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:34.966285 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8348209-28de-48cb-80fb-de5494add839 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:34.966818 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8348209-28de-48cb-80fb-de5494add839 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:34.967784 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8348209-28de-48cb-80fb-de5494add839 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:34.970212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8348209-28de-48cb-80fb-de5494add839 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.004022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f25fbc6-488c-458a-b641-2240291aca05 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.009364 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc5780a-9f52-44ec-94b9-3305e199c692 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.010309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5780a-9f52-44ec-94b9-3305e199c692 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.010664 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5780a-9f52-44ec-94b9-3305e199c692 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.011012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc5780a-9f52-44ec-94b9-3305e199c692 admin admin] PATH_INFO: `/v3/projects/797538c79be144b690f198aa4b0a6e85` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.012388 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5780a-9f52-44ec-94b9-3305e199c692 admin admin] RBAC: Authorizing `identity:delete_project(project_id=797538c79be144b690f198aa4b0a6e85)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.014403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc5780a-9f52-44ec-94b9-3305e199c692 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.026645 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 927/1854] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:34 2022] POST /identity/v3/projects => generated 383 bytes in 80 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:35.029313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8348209-28de-48cb-80fb-de5494add839 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:35.067633 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:35.104100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bbc5780a-9f52-44ec-94b9-3305e199c692 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '797538c79be144b690f198aa4b0a6e85', 'request_id': 'req-bbc5780a-9f52-44ec-94b9-3305e199c692'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:35.114909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bbc5780a-9f52-44ec-94b9-3305e199c692 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '797538c79be144b690f198aa4b0a6e85', 'request_id': 'req-bbc5780a-9f52-44ec-94b9-3305e199c692'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:35.122094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8348209-28de-48cb-80fb-de5494add839 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.123940 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bbc5780a-9f52-44ec-94b9-3305e199c692 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '797538c79be144b690f198aa4b0a6e85', 'request_id': 'req-bbc5780a-9f52-44ec-94b9-3305e199c692'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:35.140785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bbc5780a-9f52-44ec-94b9-3305e199c692 admin admin] The token cache is being invalidate because project 797538c79be144b690f198aa4b0a6e85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:35.141057 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bbc5780a-9f52-44ec-94b9-3305e199c692 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:35.142295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bbc5780a-9f52-44ec-94b9-3305e199c692 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:35.144750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 928/1855] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:34 2022] DELETE /identity/v3/projects/797538c79be144b690f198aa4b0a6e85 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:35.152200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80119b-3f8b-4dcd-a07a-b09ebb942ae7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.152658 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80119b-3f8b-4dcd-a07a-b09ebb942ae7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.153264 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80119b-3f8b-4dcd-a07a-b09ebb942ae7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.178638 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ff80119b-3f8b-4dcd-a07a-b09ebb942ae7 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:35.180347 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ff80119b-3f8b-4dcd-a07a-b09ebb942ae7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:35.190270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6747083-d9ac-457b-92b2-ac04ff69a068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.191352 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6747083-d9ac-457b-92b2-ac04ff69a068 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.191352 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6747083-d9ac-457b-92b2-ac04ff69a068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.191566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6747083-d9ac-457b-92b2-ac04ff69a068 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.192281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6747083-d9ac-457b-92b2-ac04ff69a068 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.194364 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6747083-d9ac-457b-92b2-ac04ff69a068 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.215160 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 928/1856] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:35 2022] POST /identity/v3/users => generated 292 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:35.219522 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6747083-d9ac-457b-92b2-ac04ff69a068 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:35.221203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a6747083-d9ac-457b-92b2-ac04ff69a068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:35.233052 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6747083-d9ac-457b-92b2-ac04ff69a068 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.237921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02be04-d7c8-4858-a713-4295254230e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.238598 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02be04-d7c8-4858-a713-4295254230e7 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.238909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02be04-d7c8-4858-a713-4295254230e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.239268 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a02be04-d7c8-4858-a713-4295254230e7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.240141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02be04-d7c8-4858-a713-4295254230e7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.241867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a02be04-d7c8-4858-a713-4295254230e7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.247847 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 929/1857] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:35.252286 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 929/1858] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:35.256092 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02be04-d7c8-4858-a713-4295254230e7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:35.256531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4d4ec-8bf8-4753-a048-817ceed596f1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.256972 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4d4ec-8bf8-4753-a048-817ceed596f1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.257106 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f4d4ec-8bf8-4753-a048-817ceed596f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.266209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a02be04-d7c8-4858-a713-4295254230e7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.270896 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f19c0-4a22-45ca-a8be-5f36a35355f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.271557 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f19c0-4a22-45ca-a8be-5f36a35355f5 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.271866 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f19c0-4a22-45ca-a8be-5f36a35355f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.272206 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a67f19c0-4a22-45ca-a8be-5f36a35355f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.272938 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f19c0-4a22-45ca-a8be-5f36a35355f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.274541 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a67f19c0-4a22-45ca-a8be-5f36a35355f5 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.275145 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b4f4d4ec-8bf8-4753-a048-817ceed596f1 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:35.276389 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b4f4d4ec-8bf8-4753-a048-817ceed596f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:35.284657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 930/1859] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:08:35.289330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 930/1860] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:35.289779 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4d4ec-8bf8-4753-a048-817ceed596f1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:35.292556 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f19c0-4a22-45ca-a8be-5f36a35355f5 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:35.298666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f4d4ec-8bf8-4753-a048-817ceed596f1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.303395 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32fb57-7b26-428d-a016-d7da23c28bfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.304190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32fb57-7b26-428d-a016-d7da23c28bfd admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.304483 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32fb57-7b26-428d-a016-d7da23c28bfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.304766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32fb57-7b26-428d-a016-d7da23c28bfd admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.305508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32fb57-7b26-428d-a016-d7da23c28bfd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:35.305827 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32fb57-7b26-428d-a016-d7da23c28bfd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.307613 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32fb57-7b26-428d-a016-d7da23c28bfd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.309498 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a67f19c0-4a22-45ca-a8be-5f36a35355f5 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.314310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ef36d7-6c09-483a-973a-393e7349163a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.315313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef36d7-6c09-483a-973a-393e7349163a admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.315598 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef36d7-6c09-483a-973a-393e7349163a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.316037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ef36d7-6c09-483a-973a-393e7349163a admin admin] PATH_INFO: `/v3/projects/84f4f535f32e4893a93753de97ead7b8/users/071c63c76d974ca6a1da8f4b36701009/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.329684 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 931/1861] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:08:35.330749 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32fb57-7b26-428d-a016-d7da23c28bfd admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:35.330922 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef36d7-6c09-483a-973a-393e7349163a admin admin] RBAC: Authorizing `identity:create_grant(project_id=84f4f535f32e4893a93753de97ead7b8, user_id=071c63c76d974ca6a1da8f4b36701009, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.333638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ef36d7-6c09-483a-973a-393e7349163a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.341350 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32fb57-7b26-428d-a016-d7da23c28bfd admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.348505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 931/1862] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:35 2022] PUT /identity/v3/projects/84f4f535f32e4893a93753de97ead7b8/users/071c63c76d974ca6a1da8f4b36701009/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:35.359565 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31092c7f-8816-4ee3-ae50-90cfb89bfb2c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.359878 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31092c7f-8816-4ee3-ae50-90cfb89bfb2c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.360172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31092c7f-8816-4ee3-ae50-90cfb89bfb2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.387286 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-31092c7f-8816-4ee3-ae50-90cfb89bfb2c None None] MFA Rules not processed for user `071c63c76d974ca6a1da8f4b36701009`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:35.388565 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-31092c7f-8816-4ee3-ae50-90cfb89bfb2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:35.403111 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:35.404000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07a664-0723-4c6c-b6c0-5916d60b9690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.404962 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07a664-0723-4c6c-b6c0-5916d60b9690 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.404962 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07a664-0723-4c6c-b6c0-5916d60b9690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.405240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e07a664-0723-4c6c-b6c0-5916d60b9690 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.406058 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07a664-0723-4c6c-b6c0-5916d60b9690 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.407975 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e07a664-0723-4c6c-b6c0-5916d60b9690 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.450769 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 932/1863] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:35 2022] POST /identity/v3/projects => generated 389 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:35.451844 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07a664-0723-4c6c-b6c0-5916d60b9690 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:35.471534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 932/1864] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:35 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:08:35.472653 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31092c7f-8816-4ee3-ae50-90cfb89bfb2c None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:35.473023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:35.484387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31092c7f-8816-4ee3-ae50-90cfb89bfb2c None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.536948 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e07a664-0723-4c6c-b6c0-5916d60b9690 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.542805 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4de7a-5830-4183-a1da-2008167566d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.543501 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4de7a-5830-4183-a1da-2008167566d5 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.554798 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4de7a-5830-4183-a1da-2008167566d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.555107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea4de7a-5830-4183-a1da-2008167566d5 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.555917 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4de7a-5830-4183-a1da-2008167566d5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.561631 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ea4de7a-5830-4183-a1da-2008167566d5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.575255 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a24f30-c567-4ea6-8ab5-6ab8931d9969 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.576004 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a24f30-c567-4ea6-8ab5-6ab8931d9969 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.576280 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a24f30-c567-4ea6-8ab5-6ab8931d9969 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.576619 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a24f30-c567-4ea6-8ab5-6ab8931d9969 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.577316 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a24f30-c567-4ea6-8ab5-6ab8931d9969 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.579505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84a24f30-c567-4ea6-8ab5-6ab8931d9969 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.589312 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 933/1865] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:35 2022] POST /identity/v3/users => generated 295 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:35.594570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4de7a-5830-4183-a1da-2008167566d5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:35.612092 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 933/1866] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:35 2022] POST /identity/v3/projects => generated 383 bytes in 138 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:35.614054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a24f30-c567-4ea6-8ab5-6ab8931d9969 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:35.618569 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea4de7a-5830-4183-a1da-2008167566d5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.685840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a24f30-c567-4ea6-8ab5-6ab8931d9969 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.686030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023364b8-0cef-428a-83e0-0d919b30ae37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.686895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023364b8-0cef-428a-83e0-0d919b30ae37 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.687250 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023364b8-0cef-428a-83e0-0d919b30ae37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.687600 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023364b8-0cef-428a-83e0-0d919b30ae37 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.688597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023364b8-0cef-428a-83e0-0d919b30ae37 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.690357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023364b8-0cef-428a-83e0-0d919b30ae37 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.700469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a52ab0-fe6c-4471-92eb-59a3e34f21e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.701554 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a52ab0-fe6c-4471-92eb-59a3e34f21e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.701667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 934/1867] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 108 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:35.701747 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a52ab0-fe6c-4471-92eb-59a3e34f21e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.701747 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75a52ab0-fe6c-4471-92eb-59a3e34f21e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.702579 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a52ab0-fe6c-4471-92eb-59a3e34f21e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.704926 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75a52ab0-fe6c-4471-92eb-59a3e34f21e2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.705403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023364b8-0cef-428a-83e0-0d919b30ae37 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:35.715786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023364b8-0cef-428a-83e0-0d919b30ae37 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.721079 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d8c9b-bd94-4564-a940-b6308a5dbde5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.721793 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d8c9b-bd94-4564-a940-b6308a5dbde5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.722088 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d8c9b-bd94-4564-a940-b6308a5dbde5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.722377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e3d8c9b-bd94-4564-a940-b6308a5dbde5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.723219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3d8c9b-bd94-4564-a940-b6308a5dbde5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.724928 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e3d8c9b-bd94-4564-a940-b6308a5dbde5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.730102 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 934/1868] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:35 2022] POST /identity/v3/users => generated 292 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:35.734167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a52ab0-fe6c-4471-92eb-59a3e34f21e2 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:35.744411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 935/1869] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:35.745574 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75a52ab0-fe6c-4471-92eb-59a3e34f21e2 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.746006 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d8c9b-bd94-4564-a940-b6308a5dbde5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:35.751311 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799830ce-cc9f-4b86-811b-88618521b8e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.752231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799830ce-cc9f-4b86-811b-88618521b8e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.752507 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799830ce-cc9f-4b86-811b-88618521b8e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.752790 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799830ce-cc9f-4b86-811b-88618521b8e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.753591 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799830ce-cc9f-4b86-811b-88618521b8e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.755441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799830ce-cc9f-4b86-811b-88618521b8e5 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.758286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e3d8c9b-bd94-4564-a940-b6308a5dbde5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.765123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76b84b3-461b-4340-862f-1da7559d494b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.766918 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b84b3-461b-4340-862f-1da7559d494b admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.767357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 935/1870] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:35.767891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b84b3-461b-4340-862f-1da7559d494b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.769366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76b84b3-461b-4340-862f-1da7559d494b admin admin] PATH_INFO: `/v3/projects/5d611b60a5b846e0a521caaf235da70a/users/3de28e5aebd44ff4b91c0e770a32b847/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.771409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799830ce-cc9f-4b86-811b-88618521b8e5 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:35.794232 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b84b3-461b-4340-862f-1da7559d494b admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d611b60a5b846e0a521caaf235da70a, user_id=3de28e5aebd44ff4b91c0e770a32b847, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.794447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799830ce-cc9f-4b86-811b-88618521b8e5 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.798684 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76b84b3-461b-4340-862f-1da7559d494b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.800675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64002392-0dc5-4d53-a945-26aba6e2f554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.801615 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64002392-0dc5-4d53-a945-26aba6e2f554 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.802035 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64002392-0dc5-4d53-a945-26aba6e2f554 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.802422 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64002392-0dc5-4d53-a945-26aba6e2f554 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.803332 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64002392-0dc5-4d53-a945-26aba6e2f554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.805473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64002392-0dc5-4d53-a945-26aba6e2f554 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.818075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 936/1871] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:35 2022] PUT /identity/v3/projects/5d611b60a5b846e0a521caaf235da70a/users/3de28e5aebd44ff4b91c0e770a32b847/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:35.824063 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 936/1872] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:35 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:35.824063 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64002392-0dc5-4d53-a945-26aba6e2f554 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:35.828657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222a302-0667-410a-98d7-54fefb611c14 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.829150 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222a302-0667-410a-98d7-54fefb611c14 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.829579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d222a302-0667-410a-98d7-54fefb611c14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.832794 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:35.838093 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64002392-0dc5-4d53-a945-26aba6e2f554 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:35.891123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d222a302-0667-410a-98d7-54fefb611c14 None None] MFA Rules not processed for user `3de28e5aebd44ff4b91c0e770a32b847`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:35.892682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d222a302-0667-410a-98d7-54fefb611c14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:35.938860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af3ce3-de93-4230-9d5b-30d80b671dd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:35.939999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af3ce3-de93-4230-9d5b-30d80b671dd0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:35.940293 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af3ce3-de93-4230-9d5b-30d80b671dd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:35.940577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0af3ce3-de93-4230-9d5b-30d80b671dd0 admin admin] PATH_INFO: `/v3/projects/44b0084b146d48bc9709a40c487c6a98/users/11287ae085ae4aaa8047a9c1dba1c35d/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:35.962267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af3ce3-de93-4230-9d5b-30d80b671dd0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b0084b146d48bc9709a40c487c6a98, user_id=11287ae085ae4aaa8047a9c1dba1c35d, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:35.964918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0af3ce3-de93-4230-9d5b-30d80b671dd0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:35.966055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:35.985455 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 937/1873] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:35 2022] PUT /identity/v3/projects/44b0084b146d48bc9709a40c487c6a98/users/11287ae085ae4aaa8047a9c1dba1c35d/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:35.986572 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 937/1874] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:35 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:35.991106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af3ce3-de93-4230-9d5b-30d80b671dd0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:36.005953 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26128761-d018-4edd-9a95-08e9fc109acc None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:36.006301 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26128761-d018-4edd-9a95-08e9fc109acc None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:36.006575 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26128761-d018-4edd-9a95-08e9fc109acc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:36.046943 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-26128761-d018-4edd-9a95-08e9fc109acc None None] MFA Rules not processed for user `11287ae085ae4aaa8047a9c1dba1c35d`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:36.048255 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-26128761-d018-4edd-9a95-08e9fc109acc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:36.063030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0af3ce3-de93-4230-9d5b-30d80b671dd0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:36.068298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60c62a-b12e-4937-90ac-a1c114648868 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:36.069005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60c62a-b12e-4937-90ac-a1c114648868 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:36.069298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60c62a-b12e-4937-90ac-a1c114648868 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:36.069586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60c62a-b12e-4937-90ac-a1c114648868 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:36.070294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60c62a-b12e-4937-90ac-a1c114648868 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:36.072278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60c62a-b12e-4937-90ac-a1c114648868 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:36.107125 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 938/1875] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:35 2022] POST /identity/v3/projects => generated 391 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:36.111374 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60c62a-b12e-4937-90ac-a1c114648868 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:36.126894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 938/1876] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:36 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:36.131682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26128761-d018-4edd-9a95-08e9fc109acc None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:36.148571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26128761-d018-4edd-9a95-08e9fc109acc None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:36.187533 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60c62a-b12e-4937-90ac-a1c114648868 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:36.198428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365499f-497a-42c3-8268-f897c9965972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:36.201211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365499f-497a-42c3-8268-f897c9965972 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:36.201982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365499f-497a-42c3-8268-f897c9965972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:36.202633 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7365499f-497a-42c3-8268-f897c9965972 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:36.204099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365499f-497a-42c3-8268-f897c9965972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:36.206774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7365499f-497a-42c3-8268-f897c9965972 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:36.230291 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5543d-a908-4b61-8f73-bfcc4993c0be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:36.231408 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5543d-a908-4b61-8f73-bfcc4993c0be admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:36.231953 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5543d-a908-4b61-8f73-bfcc4993c0be admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:36.232435 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92c5543d-a908-4b61-8f73-bfcc4993c0be admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:36.233369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5543d-a908-4b61-8f73-bfcc4993c0be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:36.235512 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92c5543d-a908-4b61-8f73-bfcc4993c0be admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:36.239107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 939/1877] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:36 2022] POST /identity/v3/users => generated 302 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:36.240663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365499f-497a-42c3-8268-f897c9965972 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:36.246894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:36.253944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7365499f-497a-42c3-8268-f897c9965972 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:36.261282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c0dd3-065f-446d-8f8b-f1a3d4920c5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:36.262017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c0dd3-065f-446d-8f8b-f1a3d4920c5a admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:36.262311 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c0dd3-065f-446d-8f8b-f1a3d4920c5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:36.262602 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33c0dd3-065f-446d-8f8b-f1a3d4920c5a admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:36.263349 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c0dd3-065f-446d-8f8b-f1a3d4920c5a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:36.265314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e33c0dd3-065f-446d-8f8b-f1a3d4920c5a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:36.274582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 939/1878] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:36 2022] POST /identity/v3/projects => generated 383 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:36.275493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5543d-a908-4b61-8f73-bfcc4993c0be admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:36.283179 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 940/1879] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:36.287184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c0dd3-065f-446d-8f8b-f1a3d4920c5a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:36.306461 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e33c0dd3-065f-446d-8f8b-f1a3d4920c5a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:36.371042 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92c5543d-a908-4b61-8f73-bfcc4993c0be admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:36.379233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271cd3b-a0fb-4a95-a6c2-86c32b72106c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:36.379233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271cd3b-a0fb-4a95-a6c2-86c32b72106c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:36.379233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271cd3b-a0fb-4a95-a6c2-86c32b72106c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:36.379233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a271cd3b-a0fb-4a95-a6c2-86c32b72106c admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:36.379611 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271cd3b-a0fb-4a95-a6c2-86c32b72106c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:36.383098 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a271cd3b-a0fb-4a95-a6c2-86c32b72106c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:36.390073 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5964698-2cd1-4658-8630-1a970cf4b45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:36.394679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5964698-2cd1-4658-8630-1a970cf4b45d admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:36.395056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5964698-2cd1-4658-8630-1a970cf4b45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:36.395997 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5964698-2cd1-4658-8630-1a970cf4b45d admin admin] PATH_INFO: `/v3/projects/6715b2ddf0f644d39394ef9490453077/users/8b1417f78cae4112be39f382058a7643/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:36.406210 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:36.415682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 940/1880] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:36 2022] POST /identity/v3/users => generated 298 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:36.424850 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271cd3b-a0fb-4a95-a6c2-86c32b72106c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:36.429304 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5964698-2cd1-4658-8630-1a970cf4b45d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6715b2ddf0f644d39394ef9490453077, user_id=8b1417f78cae4112be39f382058a7643, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:36.432275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5964698-2cd1-4658-8630-1a970cf4b45d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:36.440328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a271cd3b-a0fb-4a95-a6c2-86c32b72106c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:36.445208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9a88a-6c05-4653-b320-e509417ad840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:36.445890 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9a88a-6c05-4653-b320-e509417ad840 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:36.446213 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9a88a-6c05-4653-b320-e509417ad840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:36.446536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9a88a-6c05-4653-b320-e509417ad840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:36.447244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9a88a-6c05-4653-b320-e509417ad840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:36.448566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 941/1881] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:36 2022] PUT /identity/v3/projects/6715b2ddf0f644d39394ef9490453077/users/8b1417f78cae4112be39f382058a7643/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:36.448898 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9a88a-6c05-4653-b320-e509417ad840 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:36.452290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5964698-2cd1-4658-8630-1a970cf4b45d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:36.459909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 941/1882] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:36.463925 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9a88a-6c05-4653-b320-e509417ad840 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:36.473495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9a88a-6c05-4653-b320-e509417ad840 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:36.526636 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5964698-2cd1-4658-8630-1a970cf4b45d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:36.532657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641de58-b9fb-408e-996f-883934a235ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:36.532854 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa78883-2255-441b-9f59-8ef72d0ff406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:36.533020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641de58-b9fb-408e-996f-883934a235ba admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:36.533206 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa78883-2255-441b-9f59-8ef72d0ff406 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:36.533313 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641de58-b9fb-408e-996f-883934a235ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:36.533480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa78883-2255-441b-9f59-8ef72d0ff406 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:36.533585 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7641de58-b9fb-408e-996f-883934a235ba admin admin] PATH_INFO: `/v3/projects/23961c6a01db44918b65cc3118065269/users/02b194b8cfb24dd69e57024c4a89e561/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:36.534266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa78883-2255-441b-9f59-8ef72d0ff406 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:36.535029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa78883-2255-441b-9f59-8ef72d0ff406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:36.537472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa78883-2255-441b-9f59-8ef72d0ff406 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:36.552228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641de58-b9fb-408e-996f-883934a235ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=23961c6a01db44918b65cc3118065269, user_id=02b194b8cfb24dd69e57024c4a89e561, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:36.552454 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 942/1883] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:36.554204 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa78883-2255-441b-9f59-8ef72d0ff406 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:36.554311 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7641de58-b9fb-408e-996f-883934a235ba admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:36.566299 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa78883-2255-441b-9f59-8ef72d0ff406 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:36.568700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 942/1884] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:36 2022] PUT /identity/v3/projects/23961c6a01db44918b65cc3118065269/users/02b194b8cfb24dd69e57024c4a89e561/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:36.572737 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641de58-b9fb-408e-996f-883934a235ba admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:36.622050 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0672b97-fb10-42de-b5a7-f003a5cf0e80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:36.623080 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0672b97-fb10-42de-b5a7-f003a5cf0e80 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:36.623362 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0672b97-fb10-42de-b5a7-f003a5cf0e80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:36.623638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0672b97-fb10-42de-b5a7-f003a5cf0e80 admin admin] PATH_INFO: `/v3/projects/6715b2ddf0f644d39394ef9490453077/users/8b1417f78cae4112be39f382058a7643/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:36.625873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0672b97-fb10-42de-b5a7-f003a5cf0e80 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6715b2ddf0f644d39394ef9490453077, user_id=8b1417f78cae4112be39f382058a7643, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:36.629001 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0672b97-fb10-42de-b5a7-f003a5cf0e80 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:36.643402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7641de58-b9fb-408e-996f-883934a235ba admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:36.643618 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 943/1885] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:36 2022] PUT /identity/v3/projects/6715b2ddf0f644d39394ef9490453077/users/8b1417f78cae4112be39f382058a7643/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:36.655907 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67689c0-b743-4f0d-98d5-ea8d44d0f93d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:36.656203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67689c0-b743-4f0d-98d5-ea8d44d0f93d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:36.656490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c67689c0-b743-4f0d-98d5-ea8d44d0f93d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:36.665110 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:36.695915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c67689c0-b743-4f0d-98d5-ea8d44d0f93d None None] MFA Rules not processed for user `8b1417f78cae4112be39f382058a7643`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:36.697460 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c67689c0-b743-4f0d-98d5-ea8d44d0f93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:36.776629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab00f8-28df-4082-a71f-ca3311107f3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:36.779160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab00f8-28df-4082-a71f-ca3311107f3b admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:36.779772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab00f8-28df-4082-a71f-ca3311107f3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:36.780278 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bab00f8-28df-4082-a71f-ca3311107f3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:36.781274 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bab00f8-28df-4082-a71f-ca3311107f3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:36.783642 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bab00f8-28df-4082-a71f-ca3311107f3b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:36.806072 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 943/1886] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:36.807330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab00f8-28df-4082-a71f-ca3311107f3b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:36.820467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bab00f8-28df-4082-a71f-ca3311107f3b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:36.828823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff80d061-4630-479b-a611-5b4989d2cbc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:36.830015 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80d061-4630-479b-a611-5b4989d2cbc7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:36.830582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80d061-4630-479b-a611-5b4989d2cbc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:36.830936 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff80d061-4630-479b-a611-5b4989d2cbc7 admin admin] PATH_INFO: `/v3/projects/23961c6a01db44918b65cc3118065269/users/02b194b8cfb24dd69e57024c4a89e561/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:36.834104 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff80d061-4630-479b-a611-5b4989d2cbc7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=23961c6a01db44918b65cc3118065269, user_id=02b194b8cfb24dd69e57024c4a89e561, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:36.839166 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff80d061-4630-479b-a611-5b4989d2cbc7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:36.860042 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 944/1887] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:36 2022] PUT /identity/v3/projects/23961c6a01db44918b65cc3118065269/users/02b194b8cfb24dd69e57024c4a89e561/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:36.870734 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d2565-c01d-4644-9c6e-2f6e99c50c62 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:36.871084 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d2565-c01d-4644-9c6e-2f6e99c50c62 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:36.871376 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0d2565-c01d-4644-9c6e-2f6e99c50c62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:36.881668 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:36.917347 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bf0d2565-c01d-4644-9c6e-2f6e99c50c62 None None] MFA Rules not processed for user `02b194b8cfb24dd69e57024c4a89e561`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:36.917969 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 944/1888] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:36 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:36.918390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bf0d2565-c01d-4644-9c6e-2f6e99c50c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:36.930581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67689c0-b743-4f0d-98d5-ea8d44d0f93d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:36.932859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c67689c0-b743-4f0d-98d5-ea8d44d0f93d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:37.040210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c67689c0-b743-4f0d-98d5-ea8d44d0f93d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:37.063578 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 945/1889] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:36 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:08:37.096965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d2565-c01d-4644-9c6e-2f6e99c50c62 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:37.099064 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bf0d2565-c01d-4644-9c6e-2f6e99c50c62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:37.120627 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8dead-c303-4f14-b414-6ca05b5dda42 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:37.121424 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8dead-c303-4f14-b414-6ca05b5dda42 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:37.121801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8dead-c303-4f14-b414-6ca05b5dda42 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:37.122173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8dead-c303-4f14-b414-6ca05b5dda42 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:37.171319 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0d2565-c01d-4644-9c6e-2f6e99c50c62 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:37.175868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee2e50-ecae-4a54-8a5d-fb4f2675831e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:37.176551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee2e50-ecae-4a54-8a5d-fb4f2675831e service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:37.176839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee2e50-ecae-4a54-8a5d-fb4f2675831e service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:37.177119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee2e50-ecae-4a54-8a5d-fb4f2675831e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:37.207141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:37.210450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8dead-c303-4f14-b414-6ca05b5dda42 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:37.212324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8dead-c303-4f14-b414-6ca05b5dda42 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:37.222191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 945/1890] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:36 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 292 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:37.259310 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee2e50-ecae-4a54-8a5d-fb4f2675831e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:37.260817 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee2e50-ecae-4a54-8a5d-fb4f2675831e service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:37.270414 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 946/1891] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:37 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:08:37.287801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8dead-c303-4f14-b414-6ca05b5dda42 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:37.311327 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8dead-c303-4f14-b414-6ca05b5dda42 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:37.316580 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e289a45a-cab8-4c8e-a56d-98ce6d29e608 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:37.317585 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289a45a-cab8-4c8e-a56d-98ce6d29e608 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:37.318259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289a45a-cab8-4c8e-a56d-98ce6d29e608 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:37.318542 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e289a45a-cab8-4c8e-a56d-98ce6d29e608 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:37.383181 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289a45a-cab8-4c8e-a56d-98ce6d29e608 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:37.384716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e289a45a-cab8-4c8e-a56d-98ce6d29e608 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:37.396068 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 946/1892] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:37 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:37.877123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee2e50-ecae-4a54-8a5d-fb4f2675831e service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:37.878640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-88ee2e50-ecae-4a54-8a5d-fb4f2675831e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:37.926918 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:37.948276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee2e50-ecae-4a54-8a5d-fb4f2675831e service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:37.952991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed5b7e-35fa-4082-bbe3-e20f9748cd64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:37.953903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed5b7e-35fa-4082-bbe3-e20f9748cd64 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:37.954190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed5b7e-35fa-4082-bbe3-e20f9748cd64 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:37.954473 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ed5b7e-35fa-4082-bbe3-e20f9748cd64 admin admin] PATH_INFO: `/v3/users/df663d871de84844814a11a1ffdbca10` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:37.955677 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed5b7e-35fa-4082-bbe3-e20f9748cd64 admin admin] RBAC: Authorizing `identity:delete_user(user_id=df663d871de84844814a11a1ffdbca10)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:37.957681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ed5b7e-35fa-4082-bbe3-e20f9748cd64 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:38.008085 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b5ed5b7e-35fa-4082-bbe3-e20f9748cd64 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df663d871de84844814a11a1ffdbca10'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:38.024038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b5ed5b7e-35fa-4082-bbe3-e20f9748cd64 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df663d871de84844814a11a1ffdbca10'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:38.033521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b5ed5b7e-35fa-4082-bbe3-e20f9748cd64 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df663d871de84844814a11a1ffdbca10'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:38.034138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b5ed5b7e-35fa-4082-bbe3-e20f9748cd64 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df663d871de84844814a11a1ffdbca10'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:38.042080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b5ed5b7e-35fa-4082-bbe3-e20f9748cd64 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df663d871de84844814a11a1ffdbca10'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:38.048818 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 947/1893] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:37 2022] DELETE /identity/v3/users/df663d871de84844814a11a1ffdbca10 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:38.053056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c55a8d-b71f-4460-8054-f768470d3ab8 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:38.053407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c55a8d-b71f-4460-8054-f768470d3ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:38.053699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c55a8d-b71f-4460-8054-f768470d3ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:38.073867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-38c55a8d-b71f-4460-8054-f768470d3ab8 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:38.075319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-38c55a8d-b71f-4460-8054-f768470d3ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:38.168100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 947/1894] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:38.180944 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed5b7e-35fa-4082-bbe3-e20f9748cd64 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:38.182456 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b5ed5b7e-35fa-4082-bbe3-e20f9748cd64 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:38.259063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ed5b7e-35fa-4082-bbe3-e20f9748cd64 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:38.262925 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0389c-a053-4772-9e36-79c9f02b6378 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:38.263617 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0389c-a053-4772-9e36-79c9f02b6378 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:38.263997 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0389c-a053-4772-9e36-79c9f02b6378 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:38.264219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6e0389c-a053-4772-9e36-79c9f02b6378 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:38.274437 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0389c-a053-4772-9e36-79c9f02b6378 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:38.276096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6e0389c-a053-4772-9e36-79c9f02b6378 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:38.287741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 948/1895] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:38.669306 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c55a8d-b71f-4460-8054-f768470d3ab8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:38.670896 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-38c55a8d-b71f-4460-8054-f768470d3ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:38.688188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c55a8d-b71f-4460-8054-f768470d3ab8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:38.692837 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:38.694441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:38.694730 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:38.695015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4 admin admin] PATH_INFO: `/v3/projects/e7233dd5734d46e3b064c34fb4f1e244` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:38.696429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7233dd5734d46e3b064c34fb4f1e244)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:38.698361 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:38.769397 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7233dd5734d46e3b064c34fb4f1e244', 'request_id': 'req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:38.776198 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7233dd5734d46e3b064c34fb4f1e244', 'request_id': 'req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:38.787669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7233dd5734d46e3b064c34fb4f1e244', 'request_id': 'req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:38.804608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4 admin admin] The token cache is being invalidate because project e7233dd5734d46e3b064c34fb4f1e244 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:38.804978 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:38.805730 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:38.811082 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 948/1896] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:38 2022] DELETE /identity/v3/projects/e7233dd5734d46e3b064c34fb4f1e244 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:38.811793 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0389c-a053-4772-9e36-79c9f02b6378 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:38.822589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6e0389c-a053-4772-9e36-79c9f02b6378 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:38.919249 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f552dc0-dd25-4478-8388-3677daed3e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:38.921229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f552dc0-dd25-4478-8388-3677daed3e17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:38.921727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f552dc0-dd25-4478-8388-3677daed3e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:38.922239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f552dc0-dd25-4478-8388-3677daed3e17 admin admin] PATH_INFO: `/v3/users/68e3d7dde5ab4f6eb165b7f197100d74` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:38.924190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f552dc0-dd25-4478-8388-3677daed3e17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=68e3d7dde5ab4f6eb165b7f197100d74)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:38.926272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f552dc0-dd25-4478-8388-3677daed3e17 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:38.928890 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:38.930814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:38.932248 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:38.957629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:39.004931 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2f552dc0-dd25-4478-8388-3677daed3e17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68e3d7dde5ab4f6eb165b7f197100d74'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:39.011811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df8ed97-ae18-47a2-9f61-fa6a88c252e4 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:39.016056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63618414-62fe-4da7-94b4-a8bcf4751ad1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:39.017008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63618414-62fe-4da7-94b4-a8bcf4751ad1 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:39.017734 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63618414-62fe-4da7-94b4-a8bcf4751ad1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:39.018170 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63618414-62fe-4da7-94b4-a8bcf4751ad1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:39.019082 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2f552dc0-dd25-4478-8388-3677daed3e17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68e3d7dde5ab4f6eb165b7f197100d74'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:39.020414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-63618414-62fe-4da7-94b4-a8bcf4751ad1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:39.032386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2f552dc0-dd25-4478-8388-3677daed3e17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68e3d7dde5ab4f6eb165b7f197100d74'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:39.033229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2f552dc0-dd25-4478-8388-3677daed3e17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68e3d7dde5ab4f6eb165b7f197100d74'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:39.046139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2f552dc0-dd25-4478-8388-3677daed3e17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68e3d7dde5ab4f6eb165b7f197100d74'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:39.059167 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 949/1897] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:38 2022] DELETE /identity/v3/users/68e3d7dde5ab4f6eb165b7f197100d74 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:39.114304 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63618414-62fe-4da7-94b4-a8bcf4751ad1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:39.115902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63618414-62fe-4da7-94b4-a8bcf4751ad1 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:39.126458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 949/1898] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:38 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:08:39.570093 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f552dc0-dd25-4478-8388-3677daed3e17 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:39.572464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2f552dc0-dd25-4478-8388-3677daed3e17 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:39.777869 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f552dc0-dd25-4478-8388-3677daed3e17 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:39.791878 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d872a1c-fd18-4a8c-9465-b9f58de89bd2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:39.792851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d872a1c-fd18-4a8c-9465-b9f58de89bd2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:39.794439 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d872a1c-fd18-4a8c-9465-b9f58de89bd2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:39.794748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d872a1c-fd18-4a8c-9465-b9f58de89bd2 admin admin] PATH_INFO: `/v3/projects/1a62b18e2924417d81f957f1aae67e88` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:39.808396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d872a1c-fd18-4a8c-9465-b9f58de89bd2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a62b18e2924417d81f957f1aae67e88)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:39.810610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d872a1c-fd18-4a8c-9465-b9f58de89bd2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:39.952849 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5d872a1c-fd18-4a8c-9465-b9f58de89bd2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a62b18e2924417d81f957f1aae67e88', 'request_id': 'req-5d872a1c-fd18-4a8c-9465-b9f58de89bd2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:39.973864 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5d872a1c-fd18-4a8c-9465-b9f58de89bd2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a62b18e2924417d81f957f1aae67e88', 'request_id': 'req-5d872a1c-fd18-4a8c-9465-b9f58de89bd2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:39.982881 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5d872a1c-fd18-4a8c-9465-b9f58de89bd2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a62b18e2924417d81f957f1aae67e88', 'request_id': 'req-5d872a1c-fd18-4a8c-9465-b9f58de89bd2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:40.009211 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5d872a1c-fd18-4a8c-9465-b9f58de89bd2 admin admin] The token cache is being invalidate because project 1a62b18e2924417d81f957f1aae67e88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:40.011641 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5d872a1c-fd18-4a8c-9465-b9f58de89bd2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:40.014928 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5d872a1c-fd18-4a8c-9465-b9f58de89bd2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:40.020519 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 950/1899] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:39 2022] DELETE /identity/v3/projects/1a62b18e2924417d81f957f1aae67e88 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:40.025843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f384-6577-4342-9896-0ac3c5bceca0 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:40.026650 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f384-6577-4342-9896-0ac3c5bceca0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:40.026650 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b81f384-6577-4342-9896-0ac3c5bceca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:40.065827 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-1b81f384-6577-4342-9896-0ac3c5bceca0 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:40.067077 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1b81f384-6577-4342-9896-0ac3c5bceca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:40.274289 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 950/1900] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:40.282586 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38c0ba-f870-4b9d-abb3-edaa26e6cb7f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:40.283156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38c0ba-f870-4b9d-abb3-edaa26e6cb7f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:40.283446 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da38c0ba-f870-4b9d-abb3-edaa26e6cb7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:40.324015 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-da38c0ba-f870-4b9d-abb3-edaa26e6cb7f None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:40.325977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-da38c0ba-f870-4b9d-abb3-edaa26e6cb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:40.337232 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 951/1901] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:40.341701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81f384-6577-4342-9896-0ac3c5bceca0 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:40.343638 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da38c0ba-f870-4b9d-abb3-edaa26e6cb7f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:40.346190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-da38c0ba-f870-4b9d-abb3-edaa26e6cb7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:40.358853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b81f384-6577-4342-9896-0ac3c5bceca0 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:40.370756 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89641713-8bc9-4f64-824f-859329f169a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:40.372030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89641713-8bc9-4f64-824f-859329f169a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:40.372324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89641713-8bc9-4f64-824f-859329f169a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:40.372864 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89641713-8bc9-4f64-824f-859329f169a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:40.375890 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89641713-8bc9-4f64-824f-859329f169a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:40.376390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:40.376546 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89641713-8bc9-4f64-824f-859329f169a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:40.380797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89641713-8bc9-4f64-824f-859329f169a1 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:40.415273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 951/1902] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 74 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:40.421652 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89641713-8bc9-4f64-824f-859329f169a1 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:40.440094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89641713-8bc9-4f64-824f-859329f169a1 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:40.448106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76703524-e8c1-4f22-a755-a704d52c840c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:40.448946 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76703524-e8c1-4f22-a755-a704d52c840c admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:40.449440 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76703524-e8c1-4f22-a755-a704d52c840c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:40.449801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76703524-e8c1-4f22-a755-a704d52c840c admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:40.451147 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76703524-e8c1-4f22-a755-a704d52c840c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:40.454030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76703524-e8c1-4f22-a755-a704d52c840c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:40.475370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da38c0ba-f870-4b9d-abb3-edaa26e6cb7f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:40.479643 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72076c4-d29d-4b49-b9fe-3168c6841f28 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:40.480613 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72076c4-d29d-4b49-b9fe-3168c6841f28 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:40.481046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72076c4-d29d-4b49-b9fe-3168c6841f28 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:40.481642 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c72076c4-d29d-4b49-b9fe-3168c6841f28 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:40.483846 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c72076c4-d29d-4b49-b9fe-3168c6841f28 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:40.499228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 952/1903] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:40 2022] POST /identity/v3/projects => generated 385 bytes in 77 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:40.500471 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76703524-e8c1-4f22-a755-a704d52c840c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:40.548258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:40.573012 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76703524-e8c1-4f22-a755-a704d52c840c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:40.578326 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee205ae6-56cb-4dfd-b7a2-c4b2cd488d22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:40.578518 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72076c4-d29d-4b49-b9fe-3168c6841f28 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:40.579145 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee205ae6-56cb-4dfd-b7a2-c4b2cd488d22 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:40.579459 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee205ae6-56cb-4dfd-b7a2-c4b2cd488d22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:40.579845 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c72076c4-d29d-4b49-b9fe-3168c6841f28 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:40.579974 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee205ae6-56cb-4dfd-b7a2-c4b2cd488d22 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:40.580599 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee205ae6-56cb-4dfd-b7a2-c4b2cd488d22 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:40.582532 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee205ae6-56cb-4dfd-b7a2-c4b2cd488d22 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:40.590071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 952/1904] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:40 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:40.606919 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 953/1905] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:40 2022] POST /identity/v3/users => generated 293 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:40.611379 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72076c4-d29d-4b49-b9fe-3168c6841f28 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:40.622879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c72076c4-d29d-4b49-b9fe-3168c6841f28 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:40.628865 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a159f3c-b0f5-4ca4-acb9-c2898dcecd0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:40.629603 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a159f3c-b0f5-4ca4-acb9-c2898dcecd0e admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:40.629900 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a159f3c-b0f5-4ca4-acb9-c2898dcecd0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:40.630185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a159f3c-b0f5-4ca4-acb9-c2898dcecd0e admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:40.631046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a159f3c-b0f5-4ca4-acb9-c2898dcecd0e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:40.633028 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a159f3c-b0f5-4ca4-acb9-c2898dcecd0e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:40.647859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 953/1906] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:40.650356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee205ae6-56cb-4dfd-b7a2-c4b2cd488d22 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:40.662258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee205ae6-56cb-4dfd-b7a2-c4b2cd488d22 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:40.669221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e3f84a-c504-4aed-8e71-e8faa5958fda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:40.669977 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3f84a-c504-4aed-8e71-e8faa5958fda admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:40.670276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3f84a-c504-4aed-8e71-e8faa5958fda admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:40.670560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e3f84a-c504-4aed-8e71-e8faa5958fda admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:40.672053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3f84a-c504-4aed-8e71-e8faa5958fda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:40.673689 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e3f84a-c504-4aed-8e71-e8faa5958fda admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:40.685991 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 954/1907] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:40 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:40.690244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a159f3c-b0f5-4ca4-acb9-c2898dcecd0e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:40.701667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a159f3c-b0f5-4ca4-acb9-c2898dcecd0e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:40.708853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204048b-919d-461c-a5b5-7a5d634e0c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:40.710007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204048b-919d-461c-a5b5-7a5d634e0c45 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:40.710302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204048b-919d-461c-a5b5-7a5d634e0c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:40.710587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f204048b-919d-461c-a5b5-7a5d634e0c45 admin admin] PATH_INFO: `/v3/projects/7444edb096dc4c80b0b64f04736029f7/users/d7c28cde9050408ba6a90b5630b6cce2/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:40.726763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204048b-919d-461c-a5b5-7a5d634e0c45 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7444edb096dc4c80b0b64f04736029f7, user_id=d7c28cde9050408ba6a90b5630b6cce2, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:40.729719 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f204048b-919d-461c-a5b5-7a5d634e0c45 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:40.744550 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 954/1908] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:40 2022] PUT /identity/v3/projects/7444edb096dc4c80b0b64f04736029f7/users/d7c28cde9050408ba6a90b5630b6cce2/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:40.755557 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3c69d-8744-443b-9f29-2097c93887ba None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:40.755889 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3c69d-8744-443b-9f29-2097c93887ba None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:40.756242 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3c69d-8744-443b-9f29-2097c93887ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:40.782715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-36c3c69d-8744-443b-9f29-2097c93887ba None None] MFA Rules not processed for user `d7c28cde9050408ba6a90b5630b6cce2`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:40.784017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-36c3c69d-8744-443b-9f29-2097c93887ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:40.881789 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 955/1909] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:40 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:40.886675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204048b-919d-461c-a5b5-7a5d634e0c45 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:40.963584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f204048b-919d-461c-a5b5-7a5d634e0c45 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:40.971140 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca725c-312f-44b3-893f-763cefdcd937 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:40.972473 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca725c-312f-44b3-893f-763cefdcd937 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:40.972915 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca725c-312f-44b3-893f-763cefdcd937 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:40.979169 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ca725c-312f-44b3-893f-763cefdcd937 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:40.980024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca725c-312f-44b3-893f-763cefdcd937 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:40.982209 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ca725c-312f-44b3-893f-763cefdcd937 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:41.024365 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 955/1910] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:40 2022] POST /identity/v3/projects => generated 385 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:41.030809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3c69d-8744-443b-9f29-2097c93887ba None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:41.061828 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3c69d-8744-443b-9f29-2097c93887ba None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:41.137711 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c226ed-b59a-4687-9db3-5571d9b29abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:41.138529 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c226ed-b59a-4687-9db3-5571d9b29abb admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:41.138821 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c226ed-b59a-4687-9db3-5571d9b29abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:41.139100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c226ed-b59a-4687-9db3-5571d9b29abb admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:41.140152 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c226ed-b59a-4687-9db3-5571d9b29abb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:41.142774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c226ed-b59a-4687-9db3-5571d9b29abb admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:41.171903 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 956/1911] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:41 2022] POST /identity/v3/users => generated 299 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:41.173731 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca725c-312f-44b3-893f-763cefdcd937 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:41.184369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ca725c-312f-44b3-893f-763cefdcd937 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:41.189028 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84dcb5-1d6d-4258-9190-f069c2a533f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:41.189711 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84dcb5-1d6d-4258-9190-f069c2a533f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:41.190001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84dcb5-1d6d-4258-9190-f069c2a533f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:41.190285 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a84dcb5-1d6d-4258-9190-f069c2a533f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:41.191023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84dcb5-1d6d-4258-9190-f069c2a533f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:41.192699 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a84dcb5-1d6d-4258-9190-f069c2a533f0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:41.194110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:41.209580 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 956/1912] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:41.210806 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c226ed-b59a-4687-9db3-5571d9b29abb admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:41.222153 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c226ed-b59a-4687-9db3-5571d9b29abb admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:41.227643 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816ae91-125a-4ec7-bec4-f47fecbfc787 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:41.228604 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816ae91-125a-4ec7-bec4-f47fecbfc787 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:41.228893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816ae91-125a-4ec7-bec4-f47fecbfc787 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:41.229172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f816ae91-125a-4ec7-bec4-f47fecbfc787 admin admin] PATH_INFO: `/v3/projects/7f1ced3d53164101a8a54aeca9a0efee/users/38d21fc83df2430189d70cac97b754d5/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:41.251937 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816ae91-125a-4ec7-bec4-f47fecbfc787 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f1ced3d53164101a8a54aeca9a0efee, user_id=38d21fc83df2430189d70cac97b754d5, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:41.255063 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f816ae91-125a-4ec7-bec4-f47fecbfc787 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:41.275449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 957/1913] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:41 2022] PUT /identity/v3/projects/7f1ced3d53164101a8a54aeca9a0efee/users/38d21fc83df2430189d70cac97b754d5/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:41.279889 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84dcb5-1d6d-4258-9190-f069c2a533f0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:41.309140 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a84dcb5-1d6d-4258-9190-f069c2a533f0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:41.383643 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4a5f5-248d-4d40-a9ee-815f75d76789 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:41.384234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4a5f5-248d-4d40-a9ee-815f75d76789 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:41.384530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4a5f5-248d-4d40-a9ee-815f75d76789 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:41.384819 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b4a5f5-248d-4d40-a9ee-815f75d76789 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:41.385565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4a5f5-248d-4d40-a9ee-815f75d76789 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:41.387401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b4a5f5-248d-4d40-a9ee-815f75d76789 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:41.402444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 957/1914] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:41 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:41.403802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816ae91-125a-4ec7-bec4-f47fecbfc787 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:41.417025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f816ae91-125a-4ec7-bec4-f47fecbfc787 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:41.428938 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fcc067-2c57-4bdb-814d-d2524c059ed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:41.429898 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcc067-2c57-4bdb-814d-d2524c059ed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:41.430176 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcc067-2c57-4bdb-814d-d2524c059ed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:41.430449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fcc067-2c57-4bdb-814d-d2524c059ed2 admin admin] PATH_INFO: `/v3/projects/7f1ced3d53164101a8a54aeca9a0efee/users/38d21fc83df2430189d70cac97b754d5/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:41.434511 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcc067-2c57-4bdb-814d-d2524c059ed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f1ced3d53164101a8a54aeca9a0efee, user_id=38d21fc83df2430189d70cac97b754d5, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:41.437270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fcc067-2c57-4bdb-814d-d2524c059ed2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:41.439908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4a5f5-248d-4d40-a9ee-815f75d76789 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:41.442085 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-12b4a5f5-248d-4d40-a9ee-815f75d76789 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:41.444744 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:41.451949 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 958/1915] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:41 2022] PUT /identity/v3/projects/7f1ced3d53164101a8a54aeca9a0efee/users/38d21fc83df2430189d70cac97b754d5/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:41.457393 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b4a5f5-248d-4d40-a9ee-815f75d76789 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:41.463037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8fd35-fbc9-4d7f-9a1e-d85231835830 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:41.463332 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8fd35-fbc9-4d7f-9a1e-d85231835830 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:41.463613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41c8fd35-fbc9-4d7f-9a1e-d85231835830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:41.504035 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-41c8fd35-fbc9-4d7f-9a1e-d85231835830 None None] MFA Rules not processed for user `38d21fc83df2430189d70cac97b754d5`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:41.506542 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-41c8fd35-fbc9-4d7f-9a1e-d85231835830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:41.571466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbebfe5-5d22-4a6f-97c4-1b613c0b6758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:41.572689 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbebfe5-5d22-4a6f-97c4-1b613c0b6758 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:41.573544 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbebfe5-5d22-4a6f-97c4-1b613c0b6758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:41.573900 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbebfe5-5d22-4a6f-97c4-1b613c0b6758 admin admin] PATH_INFO: `/v3/users/3de28e5aebd44ff4b91c0e770a32b847` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:41.575646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbebfe5-5d22-4a6f-97c4-1b613c0b6758 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3de28e5aebd44ff4b91c0e770a32b847)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:41.586686 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbebfe5-5d22-4a6f-97c4-1b613c0b6758 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:41.630146 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 959/1916] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:41 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:41.641826 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8fd35-fbc9-4d7f-9a1e-d85231835830 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:41.643531 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-41c8fd35-fbc9-4d7f-9a1e-d85231835830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:41.652793 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-4cbebfe5-5d22-4a6f-97c4-1b613c0b6758 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3de28e5aebd44ff4b91c0e770a32b847'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:41.666520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-4cbebfe5-5d22-4a6f-97c4-1b613c0b6758 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3de28e5aebd44ff4b91c0e770a32b847'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:41.676628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-4cbebfe5-5d22-4a6f-97c4-1b613c0b6758 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3de28e5aebd44ff4b91c0e770a32b847'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:41.678601 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-4cbebfe5-5d22-4a6f-97c4-1b613c0b6758 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3de28e5aebd44ff4b91c0e770a32b847'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:41.687261 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-4cbebfe5-5d22-4a6f-97c4-1b613c0b6758 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3de28e5aebd44ff4b91c0e770a32b847'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:41.695239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 958/1917] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:41 2022] DELETE /identity/v3/users/3de28e5aebd44ff4b91c0e770a32b847 => generated 0 bytes in 256 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:41.699895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36fc3e-a4bc-4642-b814-8cfd9536f17e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:41.700194 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36fc3e-a4bc-4642-b814-8cfd9536f17e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:41.700479 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e36fc3e-a4bc-4642-b814-8cfd9536f17e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:41.714345 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41c8fd35-fbc9-4d7f-9a1e-d85231835830 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:41.720273 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6e36fc3e-a4bc-4642-b814-8cfd9536f17e None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:41.721649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6e36fc3e-a4bc-4642-b814-8cfd9536f17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:41.727210 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:41.773078 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7bf55-b95a-4cfe-b53e-f4e9f4b4ab9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:41.773847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7bf55-b95a-4cfe-b53e-f4e9f4b4ab9f service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:41.774030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7bf55-b95a-4cfe-b53e-f4e9f4b4ab9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:41.774288 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce7bf55-b95a-4cfe-b53e-f4e9f4b4ab9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:41.776079 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3ce7bf55-b95a-4cfe-b53e-f4e9f4b4ab9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:41.795964 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 959/1918] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:08:41.807337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36fc3e-a4bc-4642-b814-8cfd9536f17e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:41.809178 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6e36fc3e-a4bc-4642-b814-8cfd9536f17e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:41.833763 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:41.847831 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7bf55-b95a-4cfe-b53e-f4e9f4b4ab9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:41.849311 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ce7bf55-b95a-4cfe-b53e-f4e9f4b4ab9f service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:41.861166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 960/1919] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:41 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:41.871086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e36fc3e-a4bc-4642-b814-8cfd9536f17e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:41.878650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7966060-f8d6-4ad3-bdf5-79e86e7f3a1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:41.879579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7966060-f8d6-4ad3-bdf5-79e86e7f3a1a service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:41.880303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7966060-f8d6-4ad3-bdf5-79e86e7f3a1a service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:41.880823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7966060-f8d6-4ad3-bdf5-79e86e7f3a1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:41.893290 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7966060-f8d6-4ad3-bdf5-79e86e7f3a1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:41.894557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7966060-f8d6-4ad3-bdf5-79e86e7f3a1a service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:41.908732 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 960/1920] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:08:42.389675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7bf55-b95a-4cfe-b53e-f4e9f4b4ab9f service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:42.391276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3ce7bf55-b95a-4cfe-b53e-f4e9f4b4ab9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:42.409744 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ce7bf55-b95a-4cfe-b53e-f4e9f4b4ab9f service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:42.416542 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:42.425620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:42.425954 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:42.426248 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068 admin admin] PATH_INFO: `/v3/projects/5d611b60a5b846e0a521caaf235da70a` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:42.427660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d611b60a5b846e0a521caaf235da70a)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:42.432021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:42.488691 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d611b60a5b846e0a521caaf235da70a', 'request_id': 'req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:42.495221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d611b60a5b846e0a521caaf235da70a', 'request_id': 'req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:42.507771 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d611b60a5b846e0a521caaf235da70a', 'request_id': 'req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:42.528833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068 admin admin] The token cache is being invalidate because project 5d611b60a5b846e0a521caaf235da70a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:42.529171 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:42.531703 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:42.534056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 961/1921] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:42 2022] DELETE /identity/v3/projects/5d611b60a5b846e0a521caaf235da70a => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:42.538149 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7966060-f8d6-4ad3-bdf5-79e86e7f3a1a service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:42.558835 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7966060-f8d6-4ad3-bdf5-79e86e7f3a1a service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:42.716525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:42.718592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:42.728544 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d9fcf5-b1a5-46d1-8cb9-392b79cbf890 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:42.729839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9fcf5-b1a5-46d1-8cb9-392b79cbf890 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:42.730315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9fcf5-b1a5-46d1-8cb9-392b79cbf890 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:42.730736 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d9fcf5-b1a5-46d1-8cb9-392b79cbf890 admin admin] PATH_INFO: `/v3/users/8b1417f78cae4112be39f382058a7643` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:42.732541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9fcf5-b1a5-46d1-8cb9-392b79cbf890 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8b1417f78cae4112be39f382058a7643)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:42.739814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d9fcf5-b1a5-46d1-8cb9-392b79cbf890 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:42.811087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58d9fcf5-b1a5-46d1-8cb9-392b79cbf890 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8b1417f78cae4112be39f382058a7643'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:42.815155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c21b0f-cd5a-4975-b6e1-ab8ba2f8d068 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:42.827490 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:42.838489 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ab417-6e47-481d-8ab8-cdd16a4aef40 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:42.840170 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ab417-6e47-481d-8ab8-cdd16a4aef40 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:42.841759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ab417-6e47-481d-8ab8-cdd16a4aef40 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:42.841759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-113ab417-6e47-481d-8ab8-cdd16a4aef40 admin admin] PATH_INFO: `/v3/users/d7c28cde9050408ba6a90b5630b6cce2` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:42.842129 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ab417-6e47-481d-8ab8-cdd16a4aef40 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d7c28cde9050408ba6a90b5630b6cce2)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:42.842945 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58d9fcf5-b1a5-46d1-8cb9-392b79cbf890 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8b1417f78cae4112be39f382058a7643'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:42.844412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-113ab417-6e47-481d-8ab8-cdd16a4aef40 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:42.857660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58d9fcf5-b1a5-46d1-8cb9-392b79cbf890 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8b1417f78cae4112be39f382058a7643'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:42.858749 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58d9fcf5-b1a5-46d1-8cb9-392b79cbf890 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8b1417f78cae4112be39f382058a7643'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:42.868167 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58d9fcf5-b1a5-46d1-8cb9-392b79cbf890 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8b1417f78cae4112be39f382058a7643'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:42.875367 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 961/1922] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:42 2022] DELETE /identity/v3/users/8b1417f78cae4112be39f382058a7643 => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:42.903910 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-113ab417-6e47-481d-8ab8-cdd16a4aef40 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd7c28cde9050408ba6a90b5630b6cce2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:42.904723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-113ab417-6e47-481d-8ab8-cdd16a4aef40 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd7c28cde9050408ba6a90b5630b6cce2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:42.919863 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-113ab417-6e47-481d-8ab8-cdd16a4aef40 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd7c28cde9050408ba6a90b5630b6cce2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:42.932611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-113ab417-6e47-481d-8ab8-cdd16a4aef40 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd7c28cde9050408ba6a90b5630b6cce2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:42.933391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-113ab417-6e47-481d-8ab8-cdd16a4aef40 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd7c28cde9050408ba6a90b5630b6cce2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:42.935738 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:42.953433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 962/1923] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:42 2022] DELETE /identity/v3/users/d7c28cde9050408ba6a90b5630b6cce2 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:42.958778 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6783a7-b0c7-4a16-80b3-f77aecad6b64 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:42.959119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6783a7-b0c7-4a16-80b3-f77aecad6b64 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:42.959801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6783a7-b0c7-4a16-80b3-f77aecad6b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:42.977015 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8a6783a7-b0c7-4a16-80b3-f77aecad6b64 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:42.978249 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8a6783a7-b0c7-4a16-80b3-f77aecad6b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:43.038892 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 962/1924] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:43.048775 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ab417-6e47-481d-8ab8-cdd16a4aef40 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:43.050406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-113ab417-6e47-481d-8ab8-cdd16a4aef40 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:43.111990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-113ab417-6e47-481d-8ab8-cdd16a4aef40 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:43.115894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9e3e6-e472-41de-9d4a-d26b425a9569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:43.116565 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9e3e6-e472-41de-9d4a-d26b425a9569 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:43.116854 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9e3e6-e472-41de-9d4a-d26b425a9569 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:43.117137 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9e3e6-e472-41de-9d4a-d26b425a9569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:43.127212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9e3e6-e472-41de-9d4a-d26b425a9569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:43.128700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91b9e3e6-e472-41de-9d4a-d26b425a9569 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:43.140265 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 963/1925] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:43.706267 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6783a7-b0c7-4a16-80b3-f77aecad6b64 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:43.708698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8a6783a7-b0c7-4a16-80b3-f77aecad6b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:43.723046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6783a7-b0c7-4a16-80b3-f77aecad6b64 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:43.728591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5b3db6-0e6c-4c89-9e0d-7cf88c3f2682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:43.730005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b3db6-0e6c-4c89-9e0d-7cf88c3f2682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:43.730665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b3db6-0e6c-4c89-9e0d-7cf88c3f2682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:43.731200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5b3db6-0e6c-4c89-9e0d-7cf88c3f2682 admin admin] PATH_INFO: `/v3/projects/6715b2ddf0f644d39394ef9490453077` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:43.732754 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b3db6-0e6c-4c89-9e0d-7cf88c3f2682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6715b2ddf0f644d39394ef9490453077)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:43.734916 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5b3db6-0e6c-4c89-9e0d-7cf88c3f2682 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:43.761262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9e3e6-e472-41de-9d4a-d26b425a9569 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:43.765573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-91b9e3e6-e472-41de-9d4a-d26b425a9569 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:43.797679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9e3e6-e472-41de-9d4a-d26b425a9569 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:43.804622 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096295d-8239-484b-91b8-39fbe0771ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:43.805611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096295d-8239-484b-91b8-39fbe0771ea7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:43.807348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096295d-8239-484b-91b8-39fbe0771ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:43.807673 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a096295d-8239-484b-91b8-39fbe0771ea7 admin admin] PATH_INFO: `/v3/projects/7444edb096dc4c80b0b64f04736029f7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:43.809917 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096295d-8239-484b-91b8-39fbe0771ea7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7444edb096dc4c80b0b64f04736029f7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:43.812480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a096295d-8239-484b-91b8-39fbe0771ea7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:43.839905 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-da5b3db6-0e6c-4c89-9e0d-7cf88c3f2682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6715b2ddf0f644d39394ef9490453077', 'request_id': 'req-da5b3db6-0e6c-4c89-9e0d-7cf88c3f2682'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:43.853750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-da5b3db6-0e6c-4c89-9e0d-7cf88c3f2682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6715b2ddf0f644d39394ef9490453077', 'request_id': 'req-da5b3db6-0e6c-4c89-9e0d-7cf88c3f2682'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:43.864002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-da5b3db6-0e6c-4c89-9e0d-7cf88c3f2682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6715b2ddf0f644d39394ef9490453077', 'request_id': 'req-da5b3db6-0e6c-4c89-9e0d-7cf88c3f2682'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:43.885264 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-da5b3db6-0e6c-4c89-9e0d-7cf88c3f2682 admin admin] The token cache is being invalidate because project 6715b2ddf0f644d39394ef9490453077 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:43.885576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-da5b3db6-0e6c-4c89-9e0d-7cf88c3f2682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:43.888233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-da5b3db6-0e6c-4c89-9e0d-7cf88c3f2682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:43.891270 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 963/1926] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:43 2022] DELETE /identity/v3/projects/6715b2ddf0f644d39394ef9490453077 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:43.897476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214620a1-49ea-418f-84d6-c193e71acf70 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:43.899111 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214620a1-49ea-418f-84d6-c193e71acf70 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:43.899723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214620a1-49ea-418f-84d6-c193e71acf70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:43.912635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a096295d-8239-484b-91b8-39fbe0771ea7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7444edb096dc4c80b0b64f04736029f7', 'request_id': 'req-a096295d-8239-484b-91b8-39fbe0771ea7'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:43.925804 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a096295d-8239-484b-91b8-39fbe0771ea7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7444edb096dc4c80b0b64f04736029f7', 'request_id': 'req-a096295d-8239-484b-91b8-39fbe0771ea7'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:43.936723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-214620a1-49ea-418f-84d6-c193e71acf70 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:43.938107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-214620a1-49ea-418f-84d6-c193e71acf70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:43.941035 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a096295d-8239-484b-91b8-39fbe0771ea7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7444edb096dc4c80b0b64f04736029f7', 'request_id': 'req-a096295d-8239-484b-91b8-39fbe0771ea7'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:43.963109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a096295d-8239-484b-91b8-39fbe0771ea7 admin admin] The token cache is being invalidate because project 7444edb096dc4c80b0b64f04736029f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:43.963421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a096295d-8239-484b-91b8-39fbe0771ea7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:43.964237 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-a096295d-8239-484b-91b8-39fbe0771ea7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:43.970024 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 964/1927] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:43 2022] DELETE /identity/v3/projects/7444edb096dc4c80b0b64f04736029f7 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:43.973043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096295d-8239-484b-91b8-39fbe0771ea7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:43.974657 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a096295d-8239-484b-91b8-39fbe0771ea7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:44.007545 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=9,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:44.027867 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 964/1928] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:44.035093 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25ae3d-3ec1-4641-aa47-1c3db065c088 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:44.035391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25ae3d-3ec1-4641-aa47-1c3db065c088 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:44.035678 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc25ae3d-3ec1-4641-aa47-1c3db065c088 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:44.056327 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dc25ae3d-3ec1-4641-aa47-1c3db065c088 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:44.057537 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dc25ae3d-3ec1-4641-aa47-1c3db065c088 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:44.065437 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a096295d-8239-484b-91b8-39fbe0771ea7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:44.068055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 965/1929] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:44.071356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa90eb6-7b40-4591-afd1-0168d2ab809c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:44.071982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa90eb6-7b40-4591-afd1-0168d2ab809c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:44.072275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa90eb6-7b40-4591-afd1-0168d2ab809c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:44.072593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa90eb6-7b40-4591-afd1-0168d2ab809c admin admin] PATH_INFO: `/v3/users/38d21fc83df2430189d70cac97b754d5` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:44.072710 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25ae3d-3ec1-4641-aa47-1c3db065c088 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:44.073995 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa90eb6-7b40-4591-afd1-0168d2ab809c admin admin] RBAC: Authorizing `identity:delete_user(user_id=38d21fc83df2430189d70cac97b754d5)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:44.076005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa90eb6-7b40-4591-afd1-0168d2ab809c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:44.090912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc25ae3d-3ec1-4641-aa47-1c3db065c088 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:44.103983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42965c6d-2e04-45b4-bc3d-fe377b28aa8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:44.103983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42965c6d-2e04-45b4-bc3d-fe377b28aa8d admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:44.103983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42965c6d-2e04-45b4-bc3d-fe377b28aa8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:44.104422 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42965c6d-2e04-45b4-bc3d-fe377b28aa8d admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:44.105259 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42965c6d-2e04-45b4-bc3d-fe377b28aa8d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:44.106270 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42965c6d-2e04-45b4-bc3d-fe377b28aa8d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:44.106567 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:44.109492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42965c6d-2e04-45b4-bc3d-fe377b28aa8d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:44.130947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 966/1930] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:08:44.132023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42965c6d-2e04-45b4-bc3d-fe377b28aa8d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:44.146570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42965c6d-2e04-45b4-bc3d-fe377b28aa8d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:44.152338 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2446f-2064-4f93-b84d-e678be7b7a1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:44.154124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2446f-2064-4f93-b84d-e678be7b7a1e admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:44.154253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2446f-2064-4f93-b84d-e678be7b7a1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:44.154478 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ec2446f-2064-4f93-b84d-e678be7b7a1e admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:44.155235 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2446f-2064-4f93-b84d-e678be7b7a1e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:44.156339 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cfa90eb6-7b40-4591-afd1-0168d2ab809c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '38d21fc83df2430189d70cac97b754d5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:44.157284 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cfa90eb6-7b40-4591-afd1-0168d2ab809c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '38d21fc83df2430189d70cac97b754d5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:44.157938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ec2446f-2064-4f93-b84d-e678be7b7a1e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:44.178725 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cfa90eb6-7b40-4591-afd1-0168d2ab809c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '38d21fc83df2430189d70cac97b754d5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:44.189793 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cfa90eb6-7b40-4591-afd1-0168d2ab809c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '38d21fc83df2430189d70cac97b754d5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:44.190536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cfa90eb6-7b40-4591-afd1-0168d2ab809c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '38d21fc83df2430189d70cac97b754d5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:44.200022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 967/1931] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:44 2022] POST /identity/v3/projects => generated 381 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:44.201311 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2446f-2064-4f93-b84d-e678be7b7a1e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:44.202916 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0ec2446f-2064-4f93-b84d-e678be7b7a1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:44.204909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 965/1932] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:43 2022] DELETE /identity/v3/users/38d21fc83df2430189d70cac97b754d5 => generated 0 bytes in 232 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:44.283294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ec2446f-2064-4f93-b84d-e678be7b7a1e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:44.367221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52f495-f125-48d7-811e-8c4663b612aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:44.368135 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52f495-f125-48d7-811e-8c4663b612aa admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:44.368497 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52f495-f125-48d7-811e-8c4663b612aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:44.368848 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a52f495-f125-48d7-811e-8c4663b612aa admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:44.369700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52f495-f125-48d7-811e-8c4663b612aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:44.372020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a52f495-f125-48d7-811e-8c4663b612aa admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:44.407680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 968/1933] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:44 2022] POST /identity/v3/users => generated 291 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:44.409122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa90eb6-7b40-4591-afd1-0168d2ab809c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:44.421102 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa90eb6-7b40-4591-afd1-0168d2ab809c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:44.429423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9e549-259a-4be4-a77d-3653ee00018f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:44.430546 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9e549-259a-4be4-a77d-3653ee00018f admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:44.430913 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9e549-259a-4be4-a77d-3653ee00018f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:44.431305 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def9e549-259a-4be4-a77d-3653ee00018f admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:44.432558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9e549-259a-4be4-a77d-3653ee00018f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:44.434361 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def9e549-259a-4be4-a77d-3653ee00018f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:44.449029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 966/1934] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:44.452612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52f495-f125-48d7-811e-8c4663b612aa admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:44.464359 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a52f495-f125-48d7-811e-8c4663b612aa admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:44.469542 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0ba10-24fd-4a04-b9d2-4543e3fb4dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:44.470354 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0ba10-24fd-4a04-b9d2-4543e3fb4dd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:44.470723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0ba10-24fd-4a04-b9d2-4543e3fb4dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:44.471011 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85e0ba10-24fd-4a04-b9d2-4543e3fb4dd3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:44.471754 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0ba10-24fd-4a04-b9d2-4543e3fb4dd3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:44.473498 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85e0ba10-24fd-4a04-b9d2-4543e3fb4dd3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:44.488941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 969/1935] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:44 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:44.493238 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9e549-259a-4be4-a77d-3653ee00018f admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:44.504081 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def9e549-259a-4be4-a77d-3653ee00018f admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:44.509466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08bcd306-ebaf-4f54-b1e5-e08885078d8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:44.510409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcd306-ebaf-4f54-b1e5-e08885078d8e admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:44.510701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcd306-ebaf-4f54-b1e5-e08885078d8e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:44.510982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bcd306-ebaf-4f54-b1e5-e08885078d8e admin admin] PATH_INFO: `/v3/projects/4db65cfe4631447782cfb6f888f46954/users/3c191d99168e4dd9a7bdf896baffcd6f/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:44.534337 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bcd306-ebaf-4f54-b1e5-e08885078d8e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4db65cfe4631447782cfb6f888f46954, user_id=3c191d99168e4dd9a7bdf896baffcd6f, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:44.538243 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08bcd306-ebaf-4f54-b1e5-e08885078d8e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:44.547798 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0ba10-24fd-4a04-b9d2-4543e3fb4dd3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:44.551241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-85e0ba10-24fd-4a04-b9d2-4543e3fb4dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:44.561218 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 967/1936] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:44 2022] PUT /identity/v3/projects/4db65cfe4631447782cfb6f888f46954/users/3c191d99168e4dd9a7bdf896baffcd6f/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:44.572851 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b4460-3d55-4a1a-be66-38d4a4fbf379 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:44.573160 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b4460-3d55-4a1a-be66-38d4a4fbf379 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:44.573452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-767b4460-3d55-4a1a-be66-38d4a4fbf379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:44.587165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85e0ba10-24fd-4a04-b9d2-4543e3fb4dd3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:44.608905 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-767b4460-3d55-4a1a-be66-38d4a4fbf379 None None] MFA Rules not processed for user `3c191d99168e4dd9a7bdf896baffcd6f`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:44.610664 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-767b4460-3d55-4a1a-be66-38d4a4fbf379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:44.632100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:44.653004 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448011ba-ab94-4aab-9177-e55d217ed516 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:44.658374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448011ba-ab94-4aab-9177-e55d217ed516 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:44.658374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448011ba-ab94-4aab-9177-e55d217ed516 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:44.658374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448011ba-ab94-4aab-9177-e55d217ed516 admin admin] PATH_INFO: `/v3/projects/7f1ced3d53164101a8a54aeca9a0efee` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:44.658374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448011ba-ab94-4aab-9177-e55d217ed516 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f1ced3d53164101a8a54aeca9a0efee)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:44.658374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448011ba-ab94-4aab-9177-e55d217ed516 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:44.702352 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 968/1937] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:44 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:44.708150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b4460-3d55-4a1a-be66-38d4a4fbf379 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:44.719397 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-448011ba-ab94-4aab-9177-e55d217ed516 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f1ced3d53164101a8a54aeca9a0efee', 'request_id': 'req-448011ba-ab94-4aab-9177-e55d217ed516'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:44.722147 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-767b4460-3d55-4a1a-be66-38d4a4fbf379 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:44.728629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-448011ba-ab94-4aab-9177-e55d217ed516 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f1ced3d53164101a8a54aeca9a0efee', 'request_id': 'req-448011ba-ab94-4aab-9177-e55d217ed516'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:44.737126 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-448011ba-ab94-4aab-9177-e55d217ed516 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f1ced3d53164101a8a54aeca9a0efee', 'request_id': 'req-448011ba-ab94-4aab-9177-e55d217ed516'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:44.750786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-448011ba-ab94-4aab-9177-e55d217ed516 admin admin] The token cache is being invalidate because project 7f1ced3d53164101a8a54aeca9a0efee was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:44.751714 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-448011ba-ab94-4aab-9177-e55d217ed516 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:44.751957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-448011ba-ab94-4aab-9177-e55d217ed516 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:44.754045 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 970/1938] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:44 2022] DELETE /identity/v3/projects/7f1ced3d53164101a8a54aeca9a0efee => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:44.759962 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b266ed-6f99-478d-85e5-aaf8962ea5a1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:44.760309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b266ed-6f99-478d-85e5-aaf8962ea5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:44.760748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b266ed-6f99-478d-85e5-aaf8962ea5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:44.782289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-36b266ed-6f99-478d-85e5-aaf8962ea5a1 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:44.783402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-36b266ed-6f99-478d-85e5-aaf8962ea5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:44.792364 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:DELETE=8,keystone:SELECT=90,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:44.807776 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074af576-2bb8-41c3-8631-d0de09059b4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:44.808659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074af576-2bb8-41c3-8631-d0de09059b4c admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:44.808949 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074af576-2bb8-41c3-8631-d0de09059b4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:44.809292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-074af576-2bb8-41c3-8631-d0de09059b4c admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:44.810090 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074af576-2bb8-41c3-8631-d0de09059b4c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:44.812767 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-074af576-2bb8-41c3-8631-d0de09059b4c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:44.851757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 969/1939] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:44 2022] POST /identity/v3/projects => generated 379 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:44.856732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074af576-2bb8-41c3-8631-d0de09059b4c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:44.858811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-074af576-2bb8-41c3-8631-d0de09059b4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:44.867969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 971/1940] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:44.874118 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c512a5-0cae-4ddc-829c-9b1e5a9ce332 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:44.874633 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c512a5-0cae-4ddc-829c-9b1e5a9ce332 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:44.874916 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c512a5-0cae-4ddc-829c-9b1e5a9ce332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:44.895134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d7c512a5-0cae-4ddc-829c-9b1e5a9ce332 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:44.896614 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d7c512a5-0cae-4ddc-829c-9b1e5a9ce332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:44.957503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-074af576-2bb8-41c3-8631-d0de09059b4c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:44.964195 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e46e63-48de-41c1-9982-037e562e0628 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:44.965442 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e46e63-48de-41c1-9982-037e562e0628 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:44.966002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e46e63-48de-41c1-9982-037e562e0628 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:44.966002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e46e63-48de-41c1-9982-037e562e0628 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:44.966714 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e46e63-48de-41c1-9982-037e562e0628 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:44.969415 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e46e63-48de-41c1-9982-037e562e0628 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:44.980170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 972/1941] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:44.984737 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c512a5-0cae-4ddc-829c-9b1e5a9ce332 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:44.997685 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c512a5-0cae-4ddc-829c-9b1e5a9ce332 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.007772 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 970/1942] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:44 2022] POST /identity/v3/users => generated 296 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:45.008239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebd572-3d24-4841-a111-4425b046d9ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.008788 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e46e63-48de-41c1-9982-037e562e0628 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.009016 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebd572-3d24-4841-a111-4425b046d9ae admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.009281 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebd572-3d24-4841-a111-4425b046d9ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.009595 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efebd572-3d24-4841-a111-4425b046d9ae admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.010353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebd572-3d24-4841-a111-4425b046d9ae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:45.010686 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebd572-3d24-4841-a111-4425b046d9ae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.012470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efebd572-3d24-4841-a111-4425b046d9ae admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.020433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e46e63-48de-41c1-9982-037e562e0628 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.022373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:45.027781 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbcbde-f70c-4cc1-9e69-026347177f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.028956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbcbde-f70c-4cc1-9e69-026347177f7d admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.029254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbcbde-f70c-4cc1-9e69-026347177f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.029536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51fbcbde-f70c-4cc1-9e69-026347177f7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.030281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbcbde-f70c-4cc1-9e69-026347177f7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.032206 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51fbcbde-f70c-4cc1-9e69-026347177f7d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.044713 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 973/1943] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:08:45.045980 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebd572-3d24-4841-a111-4425b046d9ae admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.050403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 971/1944] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:45.050403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbcbde-f70c-4cc1-9e69-026347177f7d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.052944 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-efebd572-3d24-4841-a111-4425b046d9ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:45.060294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51fbcbde-f70c-4cc1-9e69-026347177f7d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.069955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d077356-3a9d-4e58-8a05-7483c49589a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.069955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d077356-3a9d-4e58-8a05-7483c49589a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.069955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d077356-3a9d-4e58-8a05-7483c49589a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.069955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d077356-3a9d-4e58-8a05-7483c49589a2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.070535 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d077356-3a9d-4e58-8a05-7483c49589a2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.074087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efebd572-3d24-4841-a111-4425b046d9ae admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.076114 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d077356-3a9d-4e58-8a05-7483c49589a2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.087287 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf5bf6-b791-40aa-ae98-3c4a1f98fb78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.088222 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf5bf6-b791-40aa-ae98-3c4a1f98fb78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.088513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf5bf6-b791-40aa-ae98-3c4a1f98fb78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.088796 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdf5bf6-b791-40aa-ae98-3c4a1f98fb78 admin admin] PATH_INFO: `/v3/users/071c63c76d974ca6a1da8f4b36701009` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.092455 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf5bf6-b791-40aa-ae98-3c4a1f98fb78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=071c63c76d974ca6a1da8f4b36701009)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.094494 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdf5bf6-b791-40aa-ae98-3c4a1f98fb78 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.119990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 972/1945] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:45 2022] POST /identity/v3/projects => generated 383 bytes in 72 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:45.120994 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d077356-3a9d-4e58-8a05-7483c49589a2 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.168662 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9cdf5bf6-b791-40aa-ae98-3c4a1f98fb78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '071c63c76d974ca6a1da8f4b36701009'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:45.191485 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9cdf5bf6-b791-40aa-ae98-3c4a1f98fb78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '071c63c76d974ca6a1da8f4b36701009'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:45.204888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9cdf5bf6-b791-40aa-ae98-3c4a1f98fb78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '071c63c76d974ca6a1da8f4b36701009'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:45.206450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9cdf5bf6-b791-40aa-ae98-3c4a1f98fb78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '071c63c76d974ca6a1da8f4b36701009'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:45.217711 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9cdf5bf6-b791-40aa-ae98-3c4a1f98fb78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '071c63c76d974ca6a1da8f4b36701009'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:45.221207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d077356-3a9d-4e58-8a05-7483c49589a2 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.226736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba4607e-9d22-40d4-80f8-a61b7fd30da4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.228071 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4607e-9d22-40d4-80f8-a61b7fd30da4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.228327 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4607e-9d22-40d4-80f8-a61b7fd30da4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.228682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba4607e-9d22-40d4-80f8-a61b7fd30da4 admin admin] PATH_INFO: `/v3/projects/0dcead8043534eddb819b5d17b7f036b/users/a356e4d170584d928fc8ca6408931bee/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.240591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 974/1946] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:45 2022] DELETE /identity/v3/users/071c63c76d974ca6a1da8f4b36701009 => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:45.241658 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf5bf6-b791-40aa-ae98-3c4a1f98fb78 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.244567 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9cdf5bf6-b791-40aa-ae98-3c4a1f98fb78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:45.256357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4607e-9d22-40d4-80f8-a61b7fd30da4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dcead8043534eddb819b5d17b7f036b, user_id=a356e4d170584d928fc8ca6408931bee, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.259398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba4607e-9d22-40d4-80f8-a61b7fd30da4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.278928 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 973/1947] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:45 2022] PUT /identity/v3/projects/0dcead8043534eddb819b5d17b7f036b/users/a356e4d170584d928fc8ca6408931bee/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:45.282551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba4f70-8cca-4b5f-8fd1-868f499bc8fe None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.282856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba4f70-8cca-4b5f-8fd1-868f499bc8fe None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.283147 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ba4f70-8cca-4b5f-8fd1-868f499bc8fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.303318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-30ba4f70-8cca-4b5f-8fd1-868f499bc8fe None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:45.304870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-30ba4f70-8cca-4b5f-8fd1-868f499bc8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:45.317307 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:45.335501 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdf5bf6-b791-40aa-ae98-3c4a1f98fb78 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.379971 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 974/1948] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:45.381093 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba4f70-8cca-4b5f-8fd1-868f499bc8fe None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.384364 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-30ba4f70-8cca-4b5f-8fd1-868f499bc8fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:45.397055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ba4f70-8cca-4b5f-8fd1-868f499bc8fe None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.403800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30af961-492d-415f-a939-9ca611b0a800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.404517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30af961-492d-415f-a939-9ca611b0a800 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.404799 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30af961-492d-415f-a939-9ca611b0a800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.405080 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e30af961-492d-415f-a939-9ca611b0a800 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.406110 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30af961-492d-415f-a939-9ca611b0a800 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.406924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae251546-09eb-498b-bb28-049fb0b28f1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.407408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e30af961-492d-415f-a939-9ca611b0a800 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.407608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae251546-09eb-498b-bb28-049fb0b28f1d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.407944 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae251546-09eb-498b-bb28-049fb0b28f1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.408233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae251546-09eb-498b-bb28-049fb0b28f1d admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.409156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae251546-09eb-498b-bb28-049fb0b28f1d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.411554 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae251546-09eb-498b-bb28-049fb0b28f1d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.418359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 975/1949] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:45.420965 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30af961-492d-415f-a939-9ca611b0a800 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.423469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e30af961-492d-415f-a939-9ca611b0a800 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:45.438749 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 975/1950] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:45 2022] POST /identity/v3/users => generated 292 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:45.439876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae251546-09eb-498b-bb28-049fb0b28f1d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.450801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae251546-09eb-498b-bb28-049fb0b28f1d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.457859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f251692-8414-47df-924f-da88772251e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.459043 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f251692-8414-47df-924f-da88772251e1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.459486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f251692-8414-47df-924f-da88772251e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.460521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f251692-8414-47df-924f-da88772251e1 admin admin] PATH_INFO: `/v3/projects/0dcead8043534eddb819b5d17b7f036b/users/a356e4d170584d928fc8ca6408931bee/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.463273 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f251692-8414-47df-924f-da88772251e1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0dcead8043534eddb819b5d17b7f036b, user_id=a356e4d170584d928fc8ca6408931bee, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.466382 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f251692-8414-47df-924f-da88772251e1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.469887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=85,keystone:DELETE=10,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:45.482104 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 976/1951] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:45 2022] PUT /identity/v3/projects/0dcead8043534eddb819b5d17b7f036b/users/a356e4d170584d928fc8ca6408931bee/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:45.483481 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f251692-8414-47df-924f-da88772251e1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.505967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e30af961-492d-415f-a939-9ca611b0a800 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.561443 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a07c4a9-2dd9-40e0-8688-46bbd21343b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.562133 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07c4a9-2dd9-40e0-8688-46bbd21343b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.562537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07c4a9-2dd9-40e0-8688-46bbd21343b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.562693 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a07c4a9-2dd9-40e0-8688-46bbd21343b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.571862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f251692-8414-47df-924f-da88772251e1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.573379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07c4a9-2dd9-40e0-8688-46bbd21343b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.574889 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a07c4a9-2dd9-40e0-8688-46bbd21343b2 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.577295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28c9c7-e2f6-4834-9f66-4db338c743e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.577988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28c9c7-e2f6-4834-9f66-4db338c743e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.578278 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28c9c7-e2f6-4834-9f66-4db338c743e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.578562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c28c9c7-e2f6-4834-9f66-4db338c743e9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.579309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c28c9c7-e2f6-4834-9f66-4db338c743e9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.581131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c28c9c7-e2f6-4834-9f66-4db338c743e9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.585890 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 976/1952] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:45.587224 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1a009-44c6-4702-b874-8a75cbd4af00 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.587561 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1a009-44c6-4702-b874-8a75cbd4af00 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.587922 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1a009-44c6-4702-b874-8a75cbd4af00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.592723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 977/1953] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:45.596550 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28c9c7-e2f6-4834-9f66-4db338c743e9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.610943 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c28c9c7-e2f6-4834-9f66-4db338c743e9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.619981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-0cf1a009-44c6-4702-b874-8a75cbd4af00 None None] MFA Rules not processed for user `a356e4d170584d928fc8ca6408931bee`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:45.621140 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b60a7a-d653-4a91-82e2-647e4cc847ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.621277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0cf1a009-44c6-4702-b874-8a75cbd4af00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:45.622027 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b60a7a-d653-4a91-82e2-647e4cc847ac admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.622418 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b60a7a-d653-4a91-82e2-647e4cc847ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.622805 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b60a7a-d653-4a91-82e2-647e4cc847ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.623988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b60a7a-d653-4a91-82e2-647e4cc847ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.625867 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b60a7a-d653-4a91-82e2-647e4cc847ac admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.638158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 978/1954] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:45 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:45.643011 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b60a7a-d653-4a91-82e2-647e4cc847ac admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.666295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b60a7a-d653-4a91-82e2-647e4cc847ac admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.675175 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d97451c-7f98-4341-a757-5fde7a2b93f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.676425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97451c-7f98-4341-a757-5fde7a2b93f3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.676821 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97451c-7f98-4341-a757-5fde7a2b93f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.677346 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d97451c-7f98-4341-a757-5fde7a2b93f3 admin admin] PATH_INFO: `/v3/projects/5588cf45bac642188ffc9fb436a6a2c2/users/757236263242487a9ff9a42b8d957d60/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.683393 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:45.698172 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97451c-7f98-4341-a757-5fde7a2b93f3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5588cf45bac642188ffc9fb436a6a2c2, user_id=757236263242487a9ff9a42b8d957d60, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.701127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d97451c-7f98-4341-a757-5fde7a2b93f3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.715323 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 977/1955] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:45 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:45.727913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 979/1956] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:45 2022] PUT /identity/v3/projects/5588cf45bac642188ffc9fb436a6a2c2/users/757236263242487a9ff9a42b8d957d60/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:45.731808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1a009-44c6-4702-b874-8a75cbd4af00 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.733492 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0cf1a009-44c6-4702-b874-8a75cbd4af00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:45.737405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa46ab-b2e8-4c36-b2f7-0a4a79276b83 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.737781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa46ab-b2e8-4c36-b2f7-0a4a79276b83 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.738142 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61aa46ab-b2e8-4c36-b2f7-0a4a79276b83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.786838 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-61aa46ab-b2e8-4c36-b2f7-0a4a79276b83 None None] MFA Rules not processed for user `757236263242487a9ff9a42b8d957d60`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:45.788795 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-61aa46ab-b2e8-4c36-b2f7-0a4a79276b83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:45.810617 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1a009-44c6-4702-b874-8a75cbd4af00 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.866718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3a024-f3f1-412f-84bb-727c3c7db5bf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.867472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3a024-f3f1-412f-84bb-727c3c7db5bf service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.867941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3a024-f3f1-412f-84bb-727c3c7db5bf service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.868636 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50d3a024-f3f1-412f-84bb-727c3c7db5bf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.870145 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 980/1957] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:45.875758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa46ab-b2e8-4c36-b2f7-0a4a79276b83 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.960138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:45.965360 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d3a024-f3f1-412f-84bb-727c3c7db5bf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.966839 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50d3a024-f3f1-412f-84bb-727c3c7db5bf service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:45.978546 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 978/1958] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:45 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:08:45.979422 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3a024-f3f1-412f-84bb-727c3c7db5bf service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:45.981219 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-50d3a024-f3f1-412f-84bb-727c3c7db5bf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:45.988261 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61aa46ab-b2e8-4c36-b2f7-0a4a79276b83 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.993576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755b537-50d2-4634-879c-ff60dee53c8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:45.994400 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b537-50d2-4634-879c-ff60dee53c8c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:45.994683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b537-50d2-4634-879c-ff60dee53c8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:45.994970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a755b537-50d2-4634-879c-ff60dee53c8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:45.995686 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755b537-50d2-4634-879c-ff60dee53c8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:45.995900 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50d3a024-f3f1-412f-84bb-727c3c7db5bf service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:45.998649 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a755b537-50d2-4634-879c-ff60dee53c8c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:46.001753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcef24-d8d2-416f-8fd8-02b14f6ef442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:46.002627 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcef24-d8d2-416f-8fd8-02b14f6ef442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:46.003009 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcef24-d8d2-416f-8fd8-02b14f6ef442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:46.003404 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-addcef24-d8d2-416f-8fd8-02b14f6ef442 admin admin] PATH_INFO: `/v3/projects/84f4f535f32e4893a93753de97ead7b8` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:46.006189 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcef24-d8d2-416f-8fd8-02b14f6ef442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=84f4f535f32e4893a93753de97ead7b8)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:46.008252 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-addcef24-d8d2-416f-8fd8-02b14f6ef442 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:46.076339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 981/1959] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:45 2022] POST /identity/v3/projects => generated 383 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:46.077184 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755b537-50d2-4634-879c-ff60dee53c8c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:46.094245 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-addcef24-d8d2-416f-8fd8-02b14f6ef442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '84f4f535f32e4893a93753de97ead7b8', 'request_id': 'req-addcef24-d8d2-416f-8fd8-02b14f6ef442'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:46.095667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:46.103699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-addcef24-d8d2-416f-8fd8-02b14f6ef442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '84f4f535f32e4893a93753de97ead7b8', 'request_id': 'req-addcef24-d8d2-416f-8fd8-02b14f6ef442'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:46.127852 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-addcef24-d8d2-416f-8fd8-02b14f6ef442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '84f4f535f32e4893a93753de97ead7b8', 'request_id': 'req-addcef24-d8d2-416f-8fd8-02b14f6ef442'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:46.156840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-addcef24-d8d2-416f-8fd8-02b14f6ef442 admin admin] The token cache is being invalidate because project 84f4f535f32e4893a93753de97ead7b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:46.157170 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-addcef24-d8d2-416f-8fd8-02b14f6ef442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:46.158445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-addcef24-d8d2-416f-8fd8-02b14f6ef442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:46.167881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 979/1960] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:45 2022] DELETE /identity/v3/projects/84f4f535f32e4893a93753de97ead7b8 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:46.168640 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcef24-d8d2-416f-8fd8-02b14f6ef442 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:46.172008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-addcef24-d8d2-416f-8fd8-02b14f6ef442 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:46.230648 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a755b537-50d2-4634-879c-ff60dee53c8c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:46.319250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-addcef24-d8d2-416f-8fd8-02b14f6ef442 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:46.325945 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632cc96-4cad-470c-82ca-02857de0aeee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:46.326868 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632cc96-4cad-470c-82ca-02857de0aeee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:46.327211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632cc96-4cad-470c-82ca-02857de0aeee admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:46.327733 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1632cc96-4cad-470c-82ca-02857de0aeee admin admin] PATH_INFO: `/v3/users/11287ae085ae4aaa8047a9c1dba1c35d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:46.331669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632cc96-4cad-470c-82ca-02857de0aeee admin admin] RBAC: Authorizing `identity:delete_user(user_id=11287ae085ae4aaa8047a9c1dba1c35d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:46.335814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1632cc96-4cad-470c-82ca-02857de0aeee admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:46.409181 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f969b57-cf53-4773-9ed6-3058fa4e285b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:46.410907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f969b57-cf53-4773-9ed6-3058fa4e285b admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:46.411404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f969b57-cf53-4773-9ed6-3058fa4e285b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:46.412070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f969b57-cf53-4773-9ed6-3058fa4e285b admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:46.413165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f969b57-cf53-4773-9ed6-3058fa4e285b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:46.415472 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f969b57-cf53-4773-9ed6-3058fa4e285b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:46.431624 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1632cc96-4cad-470c-82ca-02857de0aeee admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '11287ae085ae4aaa8047a9c1dba1c35d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:46.432453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1632cc96-4cad-470c-82ca-02857de0aeee admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '11287ae085ae4aaa8047a9c1dba1c35d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:46.453125 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1632cc96-4cad-470c-82ca-02857de0aeee admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '11287ae085ae4aaa8047a9c1dba1c35d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:46.466135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1632cc96-4cad-470c-82ca-02857de0aeee admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '11287ae085ae4aaa8047a9c1dba1c35d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:46.466978 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1632cc96-4cad-470c-82ca-02857de0aeee admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '11287ae085ae4aaa8047a9c1dba1c35d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:46.484468 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 980/1961] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:46 2022] DELETE /identity/v3/users/11287ae085ae4aaa8047a9c1dba1c35d => generated 0 bytes in 316 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:46.496009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 982/1962] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:46 2022] POST /identity/v3/users => generated 298 bytes in 417 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:46.502620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632cc96-4cad-470c-82ca-02857de0aeee admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:46.504495 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1632cc96-4cad-470c-82ca-02857de0aeee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:46.564133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f969b57-cf53-4773-9ed6-3058fa4e285b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:46.565922 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1f969b57-cf53-4773-9ed6-3058fa4e285b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:46.578050 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f969b57-cf53-4773-9ed6-3058fa4e285b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:46.580221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:DELETE=13,keystone:SELECT=85,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:46.584552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32662a3f-f755-459b-9472-d7effc1634bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:46.584702 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1632cc96-4cad-470c-82ca-02857de0aeee admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:46.585654 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32662a3f-f755-459b-9472-d7effc1634bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:46.586034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32662a3f-f755-459b-9472-d7effc1634bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:46.586376 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32662a3f-f755-459b-9472-d7effc1634bf admin admin] PATH_INFO: `/v3/users/3c191d99168e4dd9a7bdf896baffcd6f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:46.587941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32662a3f-f755-459b-9472-d7effc1634bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c191d99168e4dd9a7bdf896baffcd6f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:46.590124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32662a3f-f755-459b-9472-d7effc1634bf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:46.597555 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29353650-6051-436d-b009-24da5380af6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:46.598264 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29353650-6051-436d-b009-24da5380af6d admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:46.598553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29353650-6051-436d-b009-24da5380af6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:46.598836 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29353650-6051-436d-b009-24da5380af6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:46.599642 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29353650-6051-436d-b009-24da5380af6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:46.601302 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29353650-6051-436d-b009-24da5380af6d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:46.612417 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 981/1963] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:46.616711 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29353650-6051-436d-b009-24da5380af6d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:46.629175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29353650-6051-436d-b009-24da5380af6d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:46.634876 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e996d-a354-4209-963b-9e8077ef31cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:46.636080 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e996d-a354-4209-963b-9e8077ef31cc admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:46.636371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e996d-a354-4209-963b-9e8077ef31cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:46.636746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d90e996d-a354-4209-963b-9e8077ef31cc admin admin] PATH_INFO: `/v3/projects/b6f0a742df574bec8f2fbb557cf084ce/users/e18cc4c75e27412fa90c6d6b25f778e2/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:46.651393 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-32662a3f-f755-459b-9472-d7effc1634bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c191d99168e4dd9a7bdf896baffcd6f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:46.656850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e996d-a354-4209-963b-9e8077ef31cc admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f0a742df574bec8f2fbb557cf084ce, user_id=e18cc4c75e27412fa90c6d6b25f778e2, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:46.660126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d90e996d-a354-4209-963b-9e8077ef31cc admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:46.667309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=87,keystone:INSERT=4,keystone:DELETE=9 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:46.672262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-32662a3f-f755-459b-9472-d7effc1634bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c191d99168e4dd9a7bdf896baffcd6f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:46.679467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 982/1964] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:46 2022] PUT /identity/v3/projects/b6f0a742df574bec8f2fbb557cf084ce/users/e18cc4c75e27412fa90c6d6b25f778e2/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:46.681020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e996d-a354-4209-963b-9e8077ef31cc admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:46.688711 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-32662a3f-f755-459b-9472-d7effc1634bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c191d99168e4dd9a7bdf896baffcd6f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:46.695282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-32662a3f-f755-459b-9472-d7effc1634bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c191d99168e4dd9a7bdf896baffcd6f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:46.705018 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-32662a3f-f755-459b-9472-d7effc1634bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c191d99168e4dd9a7bdf896baffcd6f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:46.712764 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 983/1965] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:46 2022] DELETE /identity/v3/users/3c191d99168e4dd9a7bdf896baffcd6f => generated 0 bytes in 149 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:46.717571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa24d8-16cc-40f7-85a2-dbf315e95ecd None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:46.717936 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa24d8-16cc-40f7-85a2-dbf315e95ecd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:46.718383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fa24d8-16cc-40f7-85a2-dbf315e95ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:46.738897 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-22fa24d8-16cc-40f7-85a2-dbf315e95ecd None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:46.740433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-22fa24d8-16cc-40f7-85a2-dbf315e95ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:46.749037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d90e996d-a354-4209-963b-9e8077ef31cc admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:46.824144 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 984/1966] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:46.832879 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20a06c-46db-4b45-85b9-546a6f6037ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:46.833658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20a06c-46db-4b45-85b9-546a6f6037ed admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:46.833972 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20a06c-46db-4b45-85b9-546a6f6037ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:46.834257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f20a06c-46db-4b45-85b9-546a6f6037ed admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:46.835008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20a06c-46db-4b45-85b9-546a6f6037ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:46.835221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa24d8-16cc-40f7-85a2-dbf315e95ecd None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:46.836719 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f20a06c-46db-4b45-85b9-546a6f6037ed admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:46.836833 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-22fa24d8-16cc-40f7-85a2-dbf315e95ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:46.847881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 983/1967] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:46 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:46.851881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20a06c-46db-4b45-85b9-546a6f6037ed admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:46.854092 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6f20a06c-46db-4b45-85b9-546a6f6037ed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:46.873872 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f20a06c-46db-4b45-85b9-546a6f6037ed admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:46.880996 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d74b92-1d89-48ee-a356-1e8f0b940533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:46.882366 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d74b92-1d89-48ee-a356-1e8f0b940533 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:46.882722 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d74b92-1d89-48ee-a356-1e8f0b940533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:46.884641 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d74b92-1d89-48ee-a356-1e8f0b940533 admin admin] PATH_INFO: `/v3/projects/b6f0a742df574bec8f2fbb557cf084ce/users/e18cc4c75e27412fa90c6d6b25f778e2/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:46.885694 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d74b92-1d89-48ee-a356-1e8f0b940533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6f0a742df574bec8f2fbb557cf084ce, user_id=e18cc4c75e27412fa90c6d6b25f778e2, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:46.888625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d74b92-1d89-48ee-a356-1e8f0b940533 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:46.900991 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 984/1968] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:46 2022] PUT /identity/v3/projects/b6f0a742df574bec8f2fbb557cf084ce/users/e18cc4c75e27412fa90c6d6b25f778e2/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:46.901846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d74b92-1d89-48ee-a356-1e8f0b940533 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:46.902604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fa24d8-16cc-40f7-85a2-dbf315e95ecd None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:46.903852 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-23d74b92-1d89-48ee-a356-1e8f0b940533 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:46.960562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56d17436-a04a-492c-ad8e-a35cd2d7f577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:46.961233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d17436-a04a-492c-ad8e-a35cd2d7f577 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:46.961527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d17436-a04a-492c-ad8e-a35cd2d7f577 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:46.961959 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56d17436-a04a-492c-ad8e-a35cd2d7f577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:46.971538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d17436-a04a-492c-ad8e-a35cd2d7f577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:46.973089 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56d17436-a04a-492c-ad8e-a35cd2d7f577 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:46.973605 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d74b92-1d89-48ee-a356-1e8f0b940533 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:46.978343 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17eef14-4100-471f-8d7e-490fce8acf71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:46.979228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17eef14-4100-471f-8d7e-490fce8acf71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:46.979518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17eef14-4100-471f-8d7e-490fce8acf71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:46.979774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b17eef14-4100-471f-8d7e-490fce8acf71 admin admin] PATH_INFO: `/v3/projects/44b0084b146d48bc9709a40c487c6a98` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:46.981076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17eef14-4100-471f-8d7e-490fce8acf71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=44b0084b146d48bc9709a40c487c6a98)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:46.983005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b17eef14-4100-471f-8d7e-490fce8acf71 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:46.983146 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 985/1969] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:46.984445 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978ed29-2891-4028-b4e2-b57e77dfd049 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:46.984741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978ed29-2891-4028-b4e2-b57e77dfd049 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:46.984894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:46.985022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e978ed29-2891-4028-b4e2-b57e77dfd049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:47.013735 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e978ed29-2891-4028-b4e2-b57e77dfd049 None None] MFA Rules not processed for user `e18cc4c75e27412fa90c6d6b25f778e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:47.015165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e978ed29-2891-4028-b4e2-b57e77dfd049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:47.048630 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:47.067317 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b17eef14-4100-471f-8d7e-490fce8acf71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44b0084b146d48bc9709a40c487c6a98', 'request_id': 'req-b17eef14-4100-471f-8d7e-490fce8acf71'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.074964 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b17eef14-4100-471f-8d7e-490fce8acf71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44b0084b146d48bc9709a40c487c6a98', 'request_id': 'req-b17eef14-4100-471f-8d7e-490fce8acf71'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.085682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b17eef14-4100-471f-8d7e-490fce8acf71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44b0084b146d48bc9709a40c487c6a98', 'request_id': 'req-b17eef14-4100-471f-8d7e-490fce8acf71'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.100607 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b17eef14-4100-471f-8d7e-490fce8acf71 admin admin] The token cache is being invalidate because project 44b0084b146d48bc9709a40c487c6a98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:47.100713 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b17eef14-4100-471f-8d7e-490fce8acf71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.101515 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b17eef14-4100-471f-8d7e-490fce8acf71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.103221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 985/1970] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:46 2022] DELETE /identity/v3/projects/44b0084b146d48bc9709a40c487c6a98 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:47.106009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 986/1971] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:46 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:47.108913 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17eef14-4100-471f-8d7e-490fce8acf71 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:47.110536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b17eef14-4100-471f-8d7e-490fce8acf71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:47.123285 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978ed29-2891-4028-b4e2-b57e77dfd049 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:47.126609 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e978ed29-2891-4028-b4e2-b57e77dfd049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:47.190080 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b17eef14-4100-471f-8d7e-490fce8acf71 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:47.195573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e978ed29-2891-4028-b4e2-b57e77dfd049 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:47.198623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8cf36-d777-4f41-ad2c-6f0fb775f7c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:47.200208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8cf36-d777-4f41-ad2c-6f0fb775f7c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:47.200649 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8cf36-d777-4f41-ad2c-6f0fb775f7c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:47.201098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b8cf36-d777-4f41-ad2c-6f0fb775f7c7 admin admin] PATH_INFO: `/v3/users/02b194b8cfb24dd69e57024c4a89e561` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:47.202457 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8cf36-d777-4f41-ad2c-6f0fb775f7c7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=02b194b8cfb24dd69e57024c4a89e561)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:47.204846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b8cf36-d777-4f41-ad2c-6f0fb775f7c7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:47.247487 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c880d-fecc-4586-8051-2ec5e8a4d06c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:47.248206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c880d-fecc-4586-8051-2ec5e8a4d06c service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:47.248561 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c880d-fecc-4586-8051-2ec5e8a4d06c service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:47.248781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3c880d-fecc-4586-8051-2ec5e8a4d06c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:47.253591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9d3c880d-fecc-4586-8051-2ec5e8a4d06c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:47.297319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-15b8cf36-d777-4f41-ad2c-6f0fb775f7c7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '02b194b8cfb24dd69e57024c4a89e561'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.300014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-15b8cf36-d777-4f41-ad2c-6f0fb775f7c7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '02b194b8cfb24dd69e57024c4a89e561'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.320249 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-15b8cf36-d777-4f41-ad2c-6f0fb775f7c7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '02b194b8cfb24dd69e57024c4a89e561'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.334504 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-15b8cf36-d777-4f41-ad2c-6f0fb775f7c7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '02b194b8cfb24dd69e57024c4a89e561'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.335927 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-15b8cf36-d777-4f41-ad2c-6f0fb775f7c7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '02b194b8cfb24dd69e57024c4a89e561'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.352805 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 986/1972] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:47 2022] DELETE /identity/v3/users/02b194b8cfb24dd69e57024c4a89e561 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:47.362598 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c880d-fecc-4586-8051-2ec5e8a4d06c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:47.364145 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3c880d-fecc-4586-8051-2ec5e8a4d06c service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:47.377822 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 987/1973] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:47 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 255 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:47.453935 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8cf36-d777-4f41-ad2c-6f0fb775f7c7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:47.456094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-15b8cf36-d777-4f41-ad2c-6f0fb775f7c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:47.476731 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:47.481972 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c880d-fecc-4586-8051-2ec5e8a4d06c service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:47.483681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9d3c880d-fecc-4586-8051-2ec5e8a4d06c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:47.499217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3c880d-fecc-4586-8051-2ec5e8a4d06c service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:47.558853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:47.570756 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b8cf36-d777-4f41-ad2c-6f0fb775f7c7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:47.575794 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba57e5-eba3-425b-9349-06e3ecc6dedc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:47.577126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba57e5-eba3-425b-9349-06e3ecc6dedc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:47.577553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba57e5-eba3-425b-9349-06e3ecc6dedc admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:47.577670 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bba57e5-eba3-425b-9349-06e3ecc6dedc admin admin] PATH_INFO: `/v3/projects/4db65cfe4631447782cfb6f888f46954` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:47.579692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba57e5-eba3-425b-9349-06e3ecc6dedc admin admin] RBAC: Authorizing `identity:delete_project(project_id=4db65cfe4631447782cfb6f888f46954)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:47.582821 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bba57e5-eba3-425b-9349-06e3ecc6dedc admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:47.589451 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec0544-d8d9-4060-aa36-75c92e486fbc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:47.590266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec0544-d8d9-4060-aa36-75c92e486fbc service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:47.590622 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec0544-d8d9-4060-aa36-75c92e486fbc service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:47.591647 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ec0544-d8d9-4060-aa36-75c92e486fbc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:47.595516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-71ec0544-d8d9-4060-aa36-75c92e486fbc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:47.640409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2bba57e5-eba3-425b-9349-06e3ecc6dedc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4db65cfe4631447782cfb6f888f46954', 'request_id': 'req-2bba57e5-eba3-425b-9349-06e3ecc6dedc'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.648358 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2bba57e5-eba3-425b-9349-06e3ecc6dedc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4db65cfe4631447782cfb6f888f46954', 'request_id': 'req-2bba57e5-eba3-425b-9349-06e3ecc6dedc'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.661168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2bba57e5-eba3-425b-9349-06e3ecc6dedc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4db65cfe4631447782cfb6f888f46954', 'request_id': 'req-2bba57e5-eba3-425b-9349-06e3ecc6dedc'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.665900 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec0544-d8d9-4060-aa36-75c92e486fbc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:47.667377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ec0544-d8d9-4060-aa36-75c92e486fbc service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:47.674806 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2bba57e5-eba3-425b-9349-06e3ecc6dedc admin admin] The token cache is being invalidate because project 4db65cfe4631447782cfb6f888f46954 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:47.675106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2bba57e5-eba3-425b-9349-06e3ecc6dedc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.675766 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2bba57e5-eba3-425b-9349-06e3ecc6dedc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.676903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 988/1974] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:47.680008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 987/1975] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:47 2022] DELETE /identity/v3/projects/4db65cfe4631447782cfb6f888f46954 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:47.684883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec0544-d8d9-4060-aa36-75c92e486fbc service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:47.708290 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ec0544-d8d9-4060-aa36-75c92e486fbc service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:47.738293 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba57e5-eba3-425b-9349-06e3ecc6dedc admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:47.740516 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2bba57e5-eba3-425b-9349-06e3ecc6dedc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:47.832758 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bba57e5-eba3-425b-9349-06e3ecc6dedc admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:47.835595 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acac5a5d-fb86-4864-ae1e-6744e15fb323 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:47.840900 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a44d632-fb1b-43df-955a-56c3bc29c1d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:47.841820 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac5a5d-fb86-4864-ae1e-6744e15fb323 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:47.842017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44d632-fb1b-43df-955a-56c3bc29c1d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:47.842093 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44d632-fb1b-43df-955a-56c3bc29c1d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:47.842191 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac5a5d-fb86-4864-ae1e-6744e15fb323 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:47.842428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a44d632-fb1b-43df-955a-56c3bc29c1d4 admin admin] PATH_INFO: `/v3/projects/23961c6a01db44918b65cc3118065269` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:47.842542 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acac5a5d-fb86-4864-ae1e-6744e15fb323 admin admin] PATH_INFO: `/v3/users/a356e4d170584d928fc8ca6408931bee` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:47.843948 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44d632-fb1b-43df-955a-56c3bc29c1d4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=23961c6a01db44918b65cc3118065269)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:47.844059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac5a5d-fb86-4864-ae1e-6744e15fb323 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a356e4d170584d928fc8ca6408931bee)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:47.846583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acac5a5d-fb86-4864-ae1e-6744e15fb323 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:47.846703 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a44d632-fb1b-43df-955a-56c3bc29c1d4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:47.912941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:47.920854 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5a44d632-fb1b-43df-955a-56c3bc29c1d4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '23961c6a01db44918b65cc3118065269', 'request_id': 'req-5a44d632-fb1b-43df-955a-56c3bc29c1d4'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.930805 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5a44d632-fb1b-43df-955a-56c3bc29c1d4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '23961c6a01db44918b65cc3118065269', 'request_id': 'req-5a44d632-fb1b-43df-955a-56c3bc29c1d4'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.941483 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5a44d632-fb1b-43df-955a-56c3bc29c1d4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '23961c6a01db44918b65cc3118065269', 'request_id': 'req-5a44d632-fb1b-43df-955a-56c3bc29c1d4'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.985319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5a44d632-fb1b-43df-955a-56c3bc29c1d4 admin admin] The token cache is being invalidate because project 23961c6a01db44918b65cc3118065269 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:47.987305 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5a44d632-fb1b-43df-955a-56c3bc29c1d4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.989893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5a44d632-fb1b-43df-955a-56c3bc29c1d4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:47.994998 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 988/1976] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:47 2022] DELETE /identity/v3/projects/23961c6a01db44918b65cc3118065269 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:47.999750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-acac5a5d-fb86-4864-ae1e-6744e15fb323 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a356e4d170584d928fc8ca6408931bee'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:48.000809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bf04c-c5b6-4893-9913-63f309ade865 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:48.001148 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bf04c-c5b6-4893-9913-63f309ade865 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:48.001648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51bf04c-c5b6-4893-9913-63f309ade865 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:48.015314 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-acac5a5d-fb86-4864-ae1e-6744e15fb323 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a356e4d170584d928fc8ca6408931bee'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:48.034787 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-acac5a5d-fb86-4864-ae1e-6744e15fb323 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a356e4d170584d928fc8ca6408931bee'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:48.035669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-acac5a5d-fb86-4864-ae1e-6744e15fb323 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a356e4d170584d928fc8ca6408931bee'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:48.046192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-acac5a5d-fb86-4864-ae1e-6744e15fb323 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a356e4d170584d928fc8ca6408931bee'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:48.048045 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-a51bf04c-c5b6-4893-9913-63f309ade865 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:48.049735 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a51bf04c-c5b6-4893-9913-63f309ade865 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:48.053532 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 989/1977] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:47 2022] DELETE /identity/v3/users/a356e4d170584d928fc8ca6408931bee => generated 0 bytes in 369 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:48.140762 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 989/1978] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:48.150610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f23ab-8b91-4cd8-b9b2-6df75eef0d58 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:48.150916 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f23ab-8b91-4cd8-b9b2-6df75eef0d58 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:48.151256 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d98f23ab-8b91-4cd8-b9b2-6df75eef0d58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:48.166868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:48.201144 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d98f23ab-8b91-4cd8-b9b2-6df75eef0d58 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:48.202408 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d98f23ab-8b91-4cd8-b9b2-6df75eef0d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:48.222969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 990/1979] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:48.228339 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bf04c-c5b6-4893-9913-63f309ade865 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:48.248949 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51bf04c-c5b6-4893-9913-63f309ade865 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:48.257374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f23ab-8b91-4cd8-b9b2-6df75eef0d58 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:48.258891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d98f23ab-8b91-4cd8-b9b2-6df75eef0d58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:48.271683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39cd7e-285f-448d-8248-18c91b2f8af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:48.272045 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39cd7e-285f-448d-8248-18c91b2f8af3 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:48.272332 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39cd7e-285f-448d-8248-18c91b2f8af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:48.272616 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f39cd7e-285f-448d-8248-18c91b2f8af3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:48.273368 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39cd7e-285f-448d-8248-18c91b2f8af3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:48.273687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39cd7e-285f-448d-8248-18c91b2f8af3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:48.277387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d98f23ab-8b91-4cd8-b9b2-6df75eef0d58 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:48.279239 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f39cd7e-285f-448d-8248-18c91b2f8af3 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:48.316620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 990/1980] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:48.317040 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39cd7e-285f-448d-8248-18c91b2f8af3 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:48.318780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd903792-9d41-4f51-ad3f-964f4925bea3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:48.324249 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd903792-9d41-4f51-ad3f-964f4925bea3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:48.324537 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd903792-9d41-4f51-ad3f-964f4925bea3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:48.328990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd903792-9d41-4f51-ad3f-964f4925bea3 admin admin] PATH_INFO: `/v3/users/757236263242487a9ff9a42b8d957d60` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:48.335216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd903792-9d41-4f51-ad3f-964f4925bea3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=757236263242487a9ff9a42b8d957d60)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:48.337251 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd903792-9d41-4f51-ad3f-964f4925bea3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:48.340597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f39cd7e-285f-448d-8248-18c91b2f8af3 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:48.357466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fb64b-07df-40be-97eb-337f83f38c54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:48.358431 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fb64b-07df-40be-97eb-337f83f38c54 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:48.358740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fb64b-07df-40be-97eb-337f83f38c54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:48.359014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1fb64b-07df-40be-97eb-337f83f38c54 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:48.359771 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fb64b-07df-40be-97eb-337f83f38c54 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:48.361735 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1fb64b-07df-40be-97eb-337f83f38c54 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:48.407917 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 991/1981] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:48 2022] POST /identity/v3/projects => generated 385 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:48.412854 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fb64b-07df-40be-97eb-337f83f38c54 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:48.436761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dd903792-9d41-4f51-ad3f-964f4925bea3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '757236263242487a9ff9a42b8d957d60'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:48.464995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dd903792-9d41-4f51-ad3f-964f4925bea3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '757236263242487a9ff9a42b8d957d60'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:48.485396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dd903792-9d41-4f51-ad3f-964f4925bea3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '757236263242487a9ff9a42b8d957d60'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:48.486417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dd903792-9d41-4f51-ad3f-964f4925bea3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '757236263242487a9ff9a42b8d957d60'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:48.490638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1fb64b-07df-40be-97eb-337f83f38c54 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:48.496606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d4336-f699-44b6-82e3-ff1b13b2fe49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:48.497421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d4336-f699-44b6-82e3-ff1b13b2fe49 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:48.497833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d4336-f699-44b6-82e3-ff1b13b2fe49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:48.498133 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41d4336-f699-44b6-82e3-ff1b13b2fe49 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:48.498847 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dd903792-9d41-4f51-ad3f-964f4925bea3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '757236263242487a9ff9a42b8d957d60'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:48.498957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d4336-f699-44b6-82e3-ff1b13b2fe49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:48.501565 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41d4336-f699-44b6-82e3-ff1b13b2fe49 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:48.516726 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 991/1982] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:48 2022] DELETE /identity/v3/users/757236263242487a9ff9a42b8d957d60 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:48.522833 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48478008-757d-45fd-ae2b-d72169945b0c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:48.523581 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48478008-757d-45fd-ae2b-d72169945b0c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:48.524284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48478008-757d-45fd-ae2b-d72169945b0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:48.532772 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 992/1983] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:48 2022] POST /identity/v3/users => generated 293 bytes in 121 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:48.536656 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d4336-f699-44b6-82e3-ff1b13b2fe49 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:48.538916 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c41d4336-f699-44b6-82e3-ff1b13b2fe49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:48.559539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41d4336-f699-44b6-82e3-ff1b13b2fe49 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:48.572226 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-48478008-757d-45fd-ae2b-d72169945b0c None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:48.576153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-48478008-757d-45fd-ae2b-d72169945b0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:48.599802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:48.648276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729972be-3b47-48fa-b5c9-c9b85c6b507f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:48.648975 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729972be-3b47-48fa-b5c9-c9b85c6b507f admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:48.649266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729972be-3b47-48fa-b5c9-c9b85c6b507f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:48.649551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729972be-3b47-48fa-b5c9-c9b85c6b507f admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:48.650446 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729972be-3b47-48fa-b5c9-c9b85c6b507f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:48.652307 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729972be-3b47-48fa-b5c9-c9b85c6b507f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:48.668874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 993/1984] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 130 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:48.668874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729972be-3b47-48fa-b5c9-c9b85c6b507f admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:48.671141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-729972be-3b47-48fa-b5c9-c9b85c6b507f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:48.687132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729972be-3b47-48fa-b5c9-c9b85c6b507f admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:48.698540 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 992/1985] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:48.698730 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8dfebc-0cc3-43cd-9da1-02868a20f032 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:48.698938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48478008-757d-45fd-ae2b-d72169945b0c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:48.699020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8dfebc-0cc3-43cd-9da1-02868a20f032 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:48.699020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8dfebc-0cc3-43cd-9da1-02868a20f032 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:48.699020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8dfebc-0cc3-43cd-9da1-02868a20f032 admin admin] PATH_INFO: `/v3/projects/0dcead8043534eddb819b5d17b7f036b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:48.701141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8dfebc-0cc3-43cd-9da1-02868a20f032 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0dcead8043534eddb819b5d17b7f036b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:48.703100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8dfebc-0cc3-43cd-9da1-02868a20f032 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:48.709255 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48478008-757d-45fd-ae2b-d72169945b0c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:48.715461 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7233916-b1a1-422c-be74-6b2626a8e2cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:48.716222 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7233916-b1a1-422c-be74-6b2626a8e2cb admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:48.716522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7233916-b1a1-422c-be74-6b2626a8e2cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:48.717035 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7233916-b1a1-422c-be74-6b2626a8e2cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:48.718694 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7233916-b1a1-422c-be74-6b2626a8e2cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:48.720424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7233916-b1a1-422c-be74-6b2626a8e2cb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:48.743031 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 993/1986] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:48 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:48.744214 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7233916-b1a1-422c-be74-6b2626a8e2cb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:48.746259 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f7233916-b1a1-422c-be74-6b2626a8e2cb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:48.774687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ea8dfebc-0cc3-43cd-9da1-02868a20f032 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0dcead8043534eddb819b5d17b7f036b', 'request_id': 'req-ea8dfebc-0cc3-43cd-9da1-02868a20f032'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:48.781529 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ea8dfebc-0cc3-43cd-9da1-02868a20f032 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0dcead8043534eddb819b5d17b7f036b', 'request_id': 'req-ea8dfebc-0cc3-43cd-9da1-02868a20f032'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:48.791565 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ea8dfebc-0cc3-43cd-9da1-02868a20f032 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0dcead8043534eddb819b5d17b7f036b', 'request_id': 'req-ea8dfebc-0cc3-43cd-9da1-02868a20f032'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:48.805567 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ea8dfebc-0cc3-43cd-9da1-02868a20f032 admin admin] The token cache is being invalidate because project 0dcead8043534eddb819b5d17b7f036b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:48.805909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ea8dfebc-0cc3-43cd-9da1-02868a20f032 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:48.806690 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ea8dfebc-0cc3-43cd-9da1-02868a20f032 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:48.808219 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 994/1987] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:48 2022] DELETE /identity/v3/projects/0dcead8043534eddb819b5d17b7f036b => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:48.813313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8dfebc-0cc3-43cd-9da1-02868a20f032 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:48.814774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ea8dfebc-0cc3-43cd-9da1-02868a20f032 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:48.838012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7233916-b1a1-422c-be74-6b2626a8e2cb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:48.854776 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:48.881273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8dfebc-0cc3-43cd-9da1-02868a20f032 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:48.886180 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff71559-8c25-4ff4-8fa3-389f2f7a5168 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:48.887093 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff71559-8c25-4ff4-8fa3-389f2f7a5168 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:48.887398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff71559-8c25-4ff4-8fa3-389f2f7a5168 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:48.887669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff71559-8c25-4ff4-8fa3-389f2f7a5168 admin admin] PATH_INFO: `/v3/projects/fa2a5dfc856e4c2e8d120bb546bb8588/users/6b91cef758a0482b8469b05a0d0034f5/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:48.906129 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a774ec7e-8a0e-45ae-9093-dc4c2643a4f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:48.907084 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774ec7e-8a0e-45ae-9093-dc4c2643a4f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:48.907530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774ec7e-8a0e-45ae-9093-dc4c2643a4f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:48.907814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a774ec7e-8a0e-45ae-9093-dc4c2643a4f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:48.908410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff71559-8c25-4ff4-8fa3-389f2f7a5168 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa2a5dfc856e4c2e8d120bb546bb8588, user_id=6b91cef758a0482b8469b05a0d0034f5, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:48.909757 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a774ec7e-8a0e-45ae-9093-dc4c2643a4f7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:48.911395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff71559-8c25-4ff4-8fa3-389f2f7a5168 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:48.922572 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774ec7e-8a0e-45ae-9093-dc4c2643a4f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:48.924426 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a774ec7e-8a0e-45ae-9093-dc4c2643a4f7 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:48.929042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 995/1988] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:48 2022] PUT /identity/v3/projects/fa2a5dfc856e4c2e8d120bb546bb8588/users/6b91cef758a0482b8469b05a0d0034f5/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:48.930409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bad4e2-1b62-4669-a973-f57bd5614ecc None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:48.930766 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bad4e2-1b62-4669-a973-f57bd5614ecc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:48.931050 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bad4e2-1b62-4669-a973-f57bd5614ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:48.934682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 994/1989] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:48.940980 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32baa88-e734-4079-a164-8e9d6cd3abfa None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:48.941279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32baa88-e734-4079-a164-8e9d6cd3abfa None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:48.941565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32baa88-e734-4079-a164-8e9d6cd3abfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:48.950336 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-83bad4e2-1b62-4669-a973-f57bd5614ecc None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:48.952454 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-83bad4e2-1b62-4669-a973-f57bd5614ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:48.977983 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:48.978436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f32baa88-e734-4079-a164-8e9d6cd3abfa None None] MFA Rules not processed for user `6b91cef758a0482b8469b05a0d0034f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:48.980591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f32baa88-e734-4079-a164-8e9d6cd3abfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:49.043599 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 996/1990] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:49.051797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0079c5-3c2d-4c6d-993f-cc671a5af5a0 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:49.052456 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0079c5-3c2d-4c6d-993f-cc671a5af5a0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:49.054344 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be0079c5-3c2d-4c6d-993f-cc671a5af5a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:49.094138 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-be0079c5-3c2d-4c6d-993f-cc671a5af5a0 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:49.095168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-be0079c5-3c2d-4c6d-993f-cc671a5af5a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:49.124934 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 997/1991] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:08:49.130110 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0079c5-3c2d-4c6d-993f-cc671a5af5a0 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:49.156206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 995/1992] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:48 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:49.157952 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32baa88-e734-4079-a164-8e9d6cd3abfa None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:49.161569 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f32baa88-e734-4079-a164-8e9d6cd3abfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:49.168836 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be0079c5-3c2d-4c6d-993f-cc671a5af5a0 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:49.175367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632ebc7-55b7-4aaf-a63e-883d88aa2547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:49.176108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632ebc7-55b7-4aaf-a63e-883d88aa2547 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:49.176437 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632ebc7-55b7-4aaf-a63e-883d88aa2547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:49.176813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8632ebc7-55b7-4aaf-a63e-883d88aa2547 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:49.177814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8632ebc7-55b7-4aaf-a63e-883d88aa2547 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:49.178597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8632ebc7-55b7-4aaf-a63e-883d88aa2547 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:49.180577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8632ebc7-55b7-4aaf-a63e-883d88aa2547 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:49.203458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 998/1993] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:49.204207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632ebc7-55b7-4aaf-a63e-883d88aa2547 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:49.221207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8632ebc7-55b7-4aaf-a63e-883d88aa2547 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:49.228217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe66606-2921-4b1a-9dc8-bedb218874ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:49.230141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe66606-2921-4b1a-9dc8-bedb218874ac admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:49.230865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe66606-2921-4b1a-9dc8-bedb218874ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:49.231259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe66606-2921-4b1a-9dc8-bedb218874ac admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:49.232133 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe66606-2921-4b1a-9dc8-bedb218874ac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:49.234188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe66606-2921-4b1a-9dc8-bedb218874ac admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:49.262530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32baa88-e734-4079-a164-8e9d6cd3abfa None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:49.267660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693d4c9-0172-4ebf-9257-8d7ade2b3cb1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:49.268324 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693d4c9-0172-4ebf-9257-8d7ade2b3cb1 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:49.268621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693d4c9-0172-4ebf-9257-8d7ade2b3cb1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:49.268906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1693d4c9-0172-4ebf-9257-8d7ade2b3cb1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:49.275159 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 999/1994] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:49 2022] POST /identity/v3/projects => generated 385 bytes in 68 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:49.275159 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe66606-2921-4b1a-9dc8-bedb218874ac admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:49.276974 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-afe66606-2921-4b1a-9dc8-bedb218874ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:49.286678 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693d4c9-0172-4ebf-9257-8d7ade2b3cb1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:49.288339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1693d4c9-0172-4ebf-9257-8d7ade2b3cb1 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:49.315383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 996/1995] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 158 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:49.316529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693d4c9-0172-4ebf-9257-8d7ade2b3cb1 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:49.328486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1693d4c9-0172-4ebf-9257-8d7ade2b3cb1 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:49.375087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe66606-2921-4b1a-9dc8-bedb218874ac admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:49.381174 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:49.382354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:49.382706 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:49.383057 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b admin admin] PATH_INFO: `/v3/projects/5588cf45bac642188ffc9fb436a6a2c2` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:49.385265 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=5588cf45bac642188ffc9fb436a6a2c2)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:49.388117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:49.396858 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:49.414048 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b87bf8-5154-46ab-960b-a6c2ad3293e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:49.415069 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b87bf8-5154-46ab-960b-a6c2ad3293e4 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:49.415724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b87bf8-5154-46ab-960b-a6c2ad3293e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:49.416215 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b87bf8-5154-46ab-960b-a6c2ad3293e4 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:49.417110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b87bf8-5154-46ab-960b-a6c2ad3293e4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:49.420803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b87bf8-5154-46ab-960b-a6c2ad3293e4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:49.449508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 997/1996] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:49 2022] POST /identity/v3/users => generated 293 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:49.455510 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5588cf45bac642188ffc9fb436a6a2c2', 'request_id': 'req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:49.456264 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b87bf8-5154-46ab-960b-a6c2ad3293e4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:49.461981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5588cf45bac642188ffc9fb436a6a2c2', 'request_id': 'req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:49.473157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5588cf45bac642188ffc9fb436a6a2c2', 'request_id': 'req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:49.477129 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b87bf8-5154-46ab-960b-a6c2ad3293e4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:49.489761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b admin admin] The token cache is being invalidate because project 5588cf45bac642188ffc9fb436a6a2c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:49.490052 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:49.492276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:49.494619 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1000/1997] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:49 2022] DELETE /identity/v3/projects/5588cf45bac642188ffc9fb436a6a2c2 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:49.498192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:49.500901 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:49.526517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:49.557366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb0f4a-1d48-49f7-a2a2-cd7f09ed59bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:49.558072 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb0f4a-1d48-49f7-a2a2-cd7f09ed59bf admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:49.558363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb0f4a-1d48-49f7-a2a2-cd7f09ed59bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:49.558641 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bb0f4a-1d48-49f7-a2a2-cd7f09ed59bf admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:49.559422 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb0f4a-1d48-49f7-a2a2-cd7f09ed59bf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:49.561044 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bb0f4a-1d48-49f7-a2a2-cd7f09ed59bf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:49.576971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 998/1998] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 124 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:49.581106 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb0f4a-1d48-49f7-a2a2-cd7f09ed59bf admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:49.585202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d0bb0f4a-1d48-49f7-a2a2-cd7f09ed59bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:49.601266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c54a3d3-08ee-4c3e-aeaa-701754d95c8b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:49.604946 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bb0f4a-1d48-49f7-a2a2-cd7f09ed59bf admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:49.607521 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904af97a-f98a-471c-945e-8b21041bb433 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:49.608598 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904af97a-f98a-471c-945e-8b21041bb433 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:49.608962 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904af97a-f98a-471c-945e-8b21041bb433 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:49.609315 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904af97a-f98a-471c-945e-8b21041bb433 admin admin] PATH_INFO: `/v3/users/e18cc4c75e27412fa90c6d6b25f778e2` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:49.610711 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904af97a-f98a-471c-945e-8b21041bb433 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e18cc4c75e27412fa90c6d6b25f778e2)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:49.612954 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904af97a-f98a-471c-945e-8b21041bb433 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:49.614411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3161f7f-8c21-4d29-bf27-143604d2c069 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:49.615673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3161f7f-8c21-4d29-bf27-143604d2c069 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:49.616871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3161f7f-8c21-4d29-bf27-143604d2c069 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:49.617684 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3161f7f-8c21-4d29-bf27-143604d2c069 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:49.618925 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3161f7f-8c21-4d29-bf27-143604d2c069 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:49.622262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3161f7f-8c21-4d29-bf27-143604d2c069 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:49.639271 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 999/1999] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:49 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:49.643345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3161f7f-8c21-4d29-bf27-143604d2c069 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:49.663083 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3161f7f-8c21-4d29-bf27-143604d2c069 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:49.670983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9930a95-f0d9-456e-b26a-72f4f535a0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:49.672765 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9930a95-f0d9-456e-b26a-72f4f535a0d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:49.672929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9930a95-f0d9-456e-b26a-72f4f535a0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:49.673395 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9930a95-f0d9-456e-b26a-72f4f535a0d5 admin admin] PATH_INFO: `/v3/projects/8b9c6e8600c644d5ab9dc3a4de99d5fe/users/ee499874182c4affbc2a6ee8c042c992/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:49.685486 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-904af97a-f98a-471c-945e-8b21041bb433 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e18cc4c75e27412fa90c6d6b25f778e2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:49.686478 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-904af97a-f98a-471c-945e-8b21041bb433 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e18cc4c75e27412fa90c6d6b25f778e2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:49.696031 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9930a95-f0d9-456e-b26a-72f4f535a0d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8b9c6e8600c644d5ab9dc3a4de99d5fe, user_id=ee499874182c4affbc2a6ee8c042c992, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:49.698895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9930a95-f0d9-456e-b26a-72f4f535a0d5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:49.708981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-904af97a-f98a-471c-945e-8b21041bb433 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e18cc4c75e27412fa90c6d6b25f778e2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:49.719004 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1000/2000] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:49 2022] PUT /identity/v3/projects/8b9c6e8600c644d5ab9dc3a4de99d5fe/users/ee499874182c4affbc2a6ee8c042c992/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:49.728951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-904af97a-f98a-471c-945e-8b21041bb433 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e18cc4c75e27412fa90c6d6b25f778e2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:49.728951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-904af97a-f98a-471c-945e-8b21041bb433 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e18cc4c75e27412fa90c6d6b25f778e2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:49.731070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5668d-21c7-4ec5-a197-28943180fa8d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:49.731379 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5668d-21c7-4ec5-a197-28943180fa8d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:49.732180 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dd5668d-21c7-4ec5-a197-28943180fa8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:49.744012 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1001/2001] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:49 2022] DELETE /identity/v3/users/e18cc4c75e27412fa90c6d6b25f778e2 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:49.771171 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8dd5668d-21c7-4ec5-a197-28943180fa8d None None] MFA Rules not processed for user `ee499874182c4affbc2a6ee8c042c992`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:49.772811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8dd5668d-21c7-4ec5-a197-28943180fa8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:49.932914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1001/2002] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:49 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:49.945215 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904af97a-f98a-471c-945e-8b21041bb433 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:49.947489 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-904af97a-f98a-471c-945e-8b21041bb433 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:50.024496 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904af97a-f98a-471c-945e-8b21041bb433 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:50.029037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63795290-0b1f-4f9b-af7a-8f5947e02db8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:50.029922 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63795290-0b1f-4f9b-af7a-8f5947e02db8 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:50.030363 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63795290-0b1f-4f9b-af7a-8f5947e02db8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:50.030857 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63795290-0b1f-4f9b-af7a-8f5947e02db8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:50.056068 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63795290-0b1f-4f9b-af7a-8f5947e02db8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:50.058191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63795290-0b1f-4f9b-af7a-8f5947e02db8 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:50.072083 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1002/2003] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:49 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:08:50.095445 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5668d-21c7-4ec5-a197-28943180fa8d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:50.097262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8dd5668d-21c7-4ec5-a197-28943180fa8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:50.134484 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:50.190762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dd5668d-21c7-4ec5-a197-28943180fa8d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:50.196932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc779ffb-2c9a-4974-a3d4-27ea9cd0022b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:50.197904 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc779ffb-2c9a-4974-a3d4-27ea9cd0022b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:50.199812 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc779ffb-2c9a-4974-a3d4-27ea9cd0022b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:50.200163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc779ffb-2c9a-4974-a3d4-27ea9cd0022b admin admin] PATH_INFO: `/v3/users/ee499874182c4affbc2a6ee8c042c992` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:50.201638 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc779ffb-2c9a-4974-a3d4-27ea9cd0022b admin admin] RBAC: Authorizing `identity:delete_user(user_id=ee499874182c4affbc2a6ee8c042c992)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:50.203737 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc779ffb-2c9a-4974-a3d4-27ea9cd0022b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:50.214279 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63795290-0b1f-4f9b-af7a-8f5947e02db8 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:50.215725 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-63795290-0b1f-4f9b-af7a-8f5947e02db8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:50.252798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63795290-0b1f-4f9b-af7a-8f5947e02db8 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:50.264086 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcbe6d1e-68dd-4513-9521-ccd38710b7c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:50.265363 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe6d1e-68dd-4513-9521-ccd38710b7c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:50.265821 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe6d1e-68dd-4513-9521-ccd38710b7c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:50.266038 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcbe6d1e-68dd-4513-9521-ccd38710b7c9 admin admin] PATH_INFO: `/v3/projects/b6f0a742df574bec8f2fbb557cf084ce` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:50.268328 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe6d1e-68dd-4513-9521-ccd38710b7c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6f0a742df574bec8f2fbb557cf084ce)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:50.270609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcbe6d1e-68dd-4513-9521-ccd38710b7c9 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:50.301851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bc779ffb-2c9a-4974-a3d4-27ea9cd0022b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ee499874182c4affbc2a6ee8c042c992'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:50.328603 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bc779ffb-2c9a-4974-a3d4-27ea9cd0022b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ee499874182c4affbc2a6ee8c042c992'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:50.333002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:50.338921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bc779ffb-2c9a-4974-a3d4-27ea9cd0022b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ee499874182c4affbc2a6ee8c042c992'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:50.339926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bc779ffb-2c9a-4974-a3d4-27ea9cd0022b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ee499874182c4affbc2a6ee8c042c992'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:50.347859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bc779ffb-2c9a-4974-a3d4-27ea9cd0022b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ee499874182c4affbc2a6ee8c042c992'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:50.349884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bcbe6d1e-68dd-4513-9521-ccd38710b7c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6f0a742df574bec8f2fbb557cf084ce', 'request_id': 'req-bcbe6d1e-68dd-4513-9521-ccd38710b7c9'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:50.357818 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bcbe6d1e-68dd-4513-9521-ccd38710b7c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6f0a742df574bec8f2fbb557cf084ce', 'request_id': 'req-bcbe6d1e-68dd-4513-9521-ccd38710b7c9'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:50.358440 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1002/2004] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:50 2022] DELETE /identity/v3/users/ee499874182c4affbc2a6ee8c042c992 => generated 0 bytes in 260 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:50.360162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654719b0-aa66-44df-933e-109389af9fd6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:50.360467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654719b0-aa66-44df-933e-109389af9fd6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:50.360878 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654719b0-aa66-44df-933e-109389af9fd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:50.365179 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bcbe6d1e-68dd-4513-9521-ccd38710b7c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6f0a742df574bec8f2fbb557cf084ce', 'request_id': 'req-bcbe6d1e-68dd-4513-9521-ccd38710b7c9'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:50.380057 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-654719b0-aa66-44df-933e-109389af9fd6 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:50.382462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-654719b0-aa66-44df-933e-109389af9fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:50.383321 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bcbe6d1e-68dd-4513-9521-ccd38710b7c9 admin admin] The token cache is being invalidate because project b6f0a742df574bec8f2fbb557cf084ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:50.383730 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bcbe6d1e-68dd-4513-9521-ccd38710b7c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:50.385105 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bcbe6d1e-68dd-4513-9521-ccd38710b7c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:50.386943 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1003/2005] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:50 2022] DELETE /identity/v3/projects/b6f0a742df574bec8f2fbb557cf084ce => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:50.393963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c076be-8caf-4f4b-a5ec-fe9803498a15 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:50.394246 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c076be-8caf-4f4b-a5ec-fe9803498a15 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:50.394695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42c076be-8caf-4f4b-a5ec-fe9803498a15 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:50.417295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-42c076be-8caf-4f4b-a5ec-fe9803498a15 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:50.418538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-42c076be-8caf-4f4b-a5ec-fe9803498a15 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:50.455087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1003/2006] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:50.465844 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654719b0-aa66-44df-933e-109389af9fd6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:50.468417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-654719b0-aa66-44df-933e-109389af9fd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:50.500282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1004/2007] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:50.512620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51348b89-9107-433f-9ea7-a6e1d86f4a96 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:50.512620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51348b89-9107-433f-9ea7-a6e1d86f4a96 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:50.512620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51348b89-9107-433f-9ea7-a6e1d86f4a96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:50.533614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-51348b89-9107-433f-9ea7-a6e1d86f4a96 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:50.534985 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-51348b89-9107-433f-9ea7-a6e1d86f4a96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:50.555611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1005/2008] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:50.559344 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51348b89-9107-433f-9ea7-a6e1d86f4a96 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:50.571000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654719b0-aa66-44df-933e-109389af9fd6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:50.571151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51348b89-9107-433f-9ea7-a6e1d86f4a96 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:50.576627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251385ba-c678-43b8-b8ba-fbbfa0e841a8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:50.577049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5e17a-7e0b-439e-8886-02374b6c38f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:50.577451 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251385ba-c678-43b8-b8ba-fbbfa0e841a8 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:50.577811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251385ba-c678-43b8-b8ba-fbbfa0e841a8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:50.577922 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5e17a-7e0b-439e-8886-02374b6c38f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:50.578218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-251385ba-c678-43b8-b8ba-fbbfa0e841a8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:50.578324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5e17a-7e0b-439e-8886-02374b6c38f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:50.578638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eb5e17a-7e0b-439e-8886-02374b6c38f3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:50.579461 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5e17a-7e0b-439e-8886-02374b6c38f3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:50.579864 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5e17a-7e0b-439e-8886-02374b6c38f3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:50.581576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eb5e17a-7e0b-439e-8886-02374b6c38f3 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:50.585280 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:50.589402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251385ba-c678-43b8-b8ba-fbbfa0e841a8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:50.590965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-251385ba-c678-43b8-b8ba-fbbfa0e841a8 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:50.598196 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1006/2009] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:08:50.602822 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5e17a-7e0b-439e-8886-02374b6c38f3 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:50.603951 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1004/2010] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:08:50.616519 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eb5e17a-7e0b-439e-8886-02374b6c38f3 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:50.621169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538a960-c68e-4356-a790-e5090439a0b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:50.621880 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538a960-c68e-4356-a790-e5090439a0b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:50.622182 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538a960-c68e-4356-a790-e5090439a0b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:50.622467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7538a960-c68e-4356-a790-e5090439a0b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:50.623173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538a960-c68e-4356-a790-e5090439a0b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:50.625172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7538a960-c68e-4356-a790-e5090439a0b0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:50.659362 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1007/2011] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:50 2022] POST /identity/v3/projects => generated 369 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:50.660725 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251385ba-c678-43b8-b8ba-fbbfa0e841a8 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:50.671095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-251385ba-c678-43b8-b8ba-fbbfa0e841a8 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:50.744092 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b764c7-eb48-4b10-b203-2ce4a083d689 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:50.745423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b764c7-eb48-4b10-b203-2ce4a083d689 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:50.745423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b764c7-eb48-4b10-b203-2ce4a083d689 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:50.745646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85b764c7-eb48-4b10-b203-2ce4a083d689 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:50.747174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b764c7-eb48-4b10-b203-2ce4a083d689 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:50.750356 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85b764c7-eb48-4b10-b203-2ce4a083d689 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:50.789699 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1005/2012] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:50 2022] POST /identity/v3/users => generated 285 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:50.791240 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538a960-c68e-4356-a790-e5090439a0b0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:50.809152 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7538a960-c68e-4356-a790-e5090439a0b0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:50.820846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4820f-8ead-4bc1-8b18-7bb16352cc94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:50.821734 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4820f-8ead-4bc1-8b18-7bb16352cc94 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:50.822065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4820f-8ead-4bc1-8b18-7bb16352cc94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:50.822352 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba4820f-8ead-4bc1-8b18-7bb16352cc94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:50.823161 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4820f-8ead-4bc1-8b18-7bb16352cc94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:50.824887 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba4820f-8ead-4bc1-8b18-7bb16352cc94 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:50.835983 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1008/2013] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:50.840102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b764c7-eb48-4b10-b203-2ce4a083d689 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:50.852724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85b764c7-eb48-4b10-b203-2ce4a083d689 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:50.857767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098a8875-5ae1-4664-9cdd-25dc6f5b840c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:50.858679 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a8875-5ae1-4664-9cdd-25dc6f5b840c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:50.859314 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a8875-5ae1-4664-9cdd-25dc6f5b840c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:50.859810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098a8875-5ae1-4664-9cdd-25dc6f5b840c admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:50.861029 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a8875-5ae1-4664-9cdd-25dc6f5b840c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:50.863226 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098a8875-5ae1-4664-9cdd-25dc6f5b840c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:50.881423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1006/2014] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:50.885276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4820f-8ead-4bc1-8b18-7bb16352cc94 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:50.898901 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba4820f-8ead-4bc1-8b18-7bb16352cc94 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:50.904653 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66626fd9-7b29-4d1b-8c84-902621e660e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:50.905576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66626fd9-7b29-4d1b-8c84-902621e660e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:50.905964 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66626fd9-7b29-4d1b-8c84-902621e660e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:50.906291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66626fd9-7b29-4d1b-8c84-902621e660e4 admin admin] PATH_INFO: `/v3/projects/e37f7240a9bd4f36a63f9c6043aecc88/users/b2459b9a6a3c495a82702b184656f664/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:50.938310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66626fd9-7b29-4d1b-8c84-902621e660e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e37f7240a9bd4f36a63f9c6043aecc88, user_id=b2459b9a6a3c495a82702b184656f664, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:50.941056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66626fd9-7b29-4d1b-8c84-902621e660e4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:50.972363 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1009/2015] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:50 2022] PUT /identity/v3/projects/e37f7240a9bd4f36a63f9c6043aecc88/users/b2459b9a6a3c495a82702b184656f664/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:50.983978 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efcf15-6140-4f19-9265-151e80540d2d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:50.984483 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efcf15-6140-4f19-9265-151e80540d2d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:50.984586 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25efcf15-6140-4f19-9265-151e80540d2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:51.040702 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-25efcf15-6140-4f19-9265-151e80540d2d None None] MFA Rules not processed for user `b2459b9a6a3c495a82702b184656f664`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:51.041937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-25efcf15-6140-4f19-9265-151e80540d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:51.155850 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1007/2016] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:50 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:51.169640 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66626fd9-7b29-4d1b-8c84-902621e660e4 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:51.171491 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-66626fd9-7b29-4d1b-8c84-902621e660e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:51.232367 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efcf15-6140-4f19-9265-151e80540d2d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:51.233796 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-25efcf15-6140-4f19-9265-151e80540d2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:51.276406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:51.303597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66626fd9-7b29-4d1b-8c84-902621e660e4 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:51.308511 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79aef07a-ee0b-4121-b06a-3e0a599ec7a9 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '196dc406a95444c39babc50d59f9eb6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:51.310520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aef07a-ee0b-4121-b06a-3e0a599ec7a9 service glance] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:51.317109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aef07a-ee0b-4121-b06a-3e0a599ec7a9 service glance] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:51.317525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79aef07a-ee0b-4121-b06a-3e0a599ec7a9 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:51.340996 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aef07a-ee0b-4121-b06a-3e0a599ec7a9 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:51.342636 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79aef07a-ee0b-4121-b06a-3e0a599ec7a9 service glance] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:51.361952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1010/2017] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:51 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:51.373828 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25efcf15-6140-4f19-9265-151e80540d2d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:51.382098 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-564688b2-1099-4303-bbb4-12c6b0d73ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:51.382997 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564688b2-1099-4303-bbb4-12c6b0d73ff6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:51.383291 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564688b2-1099-4303-bbb4-12c6b0d73ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:51.386363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-564688b2-1099-4303-bbb4-12c6b0d73ff6 admin admin] PATH_INFO: `/v3/projects/8b9c6e8600c644d5ab9dc3a4de99d5fe` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:51.388363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564688b2-1099-4303-bbb4-12c6b0d73ff6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8b9c6e8600c644d5ab9dc3a4de99d5fe)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:51.409417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-564688b2-1099-4303-bbb4-12c6b0d73ff6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:51.412460 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:51.494432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-564688b2-1099-4303-bbb4-12c6b0d73ff6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8b9c6e8600c644d5ab9dc3a4de99d5fe', 'request_id': 'req-564688b2-1099-4303-bbb4-12c6b0d73ff6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:51.508411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-564688b2-1099-4303-bbb4-12c6b0d73ff6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8b9c6e8600c644d5ab9dc3a4de99d5fe', 'request_id': 'req-564688b2-1099-4303-bbb4-12c6b0d73ff6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:51.524405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-564688b2-1099-4303-bbb4-12c6b0d73ff6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8b9c6e8600c644d5ab9dc3a4de99d5fe', 'request_id': 'req-564688b2-1099-4303-bbb4-12c6b0d73ff6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:51.559749 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-564688b2-1099-4303-bbb4-12c6b0d73ff6 admin admin] The token cache is being invalidate because project 8b9c6e8600c644d5ab9dc3a4de99d5fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:51.560188 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-564688b2-1099-4303-bbb4-12c6b0d73ff6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:51.560905 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-564688b2-1099-4303-bbb4-12c6b0d73ff6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:51.562709 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1008/2018] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:51 2022] DELETE /identity/v3/projects/8b9c6e8600c644d5ab9dc3a4de99d5fe => generated 0 bytes in 331 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:51.568403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa170b7-8475-4695-95da-54dcd0bd8293 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:51.569186 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa170b7-8475-4695-95da-54dcd0bd8293 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:51.569602 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa170b7-8475-4695-95da-54dcd0bd8293 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:51.601118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-2aa170b7-8475-4695-95da-54dcd0bd8293 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:51.602354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2aa170b7-8475-4695-95da-54dcd0bd8293 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:51.729564 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1011/2019] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:51.732199 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7adb6c-ae41-4f0f-92fa-8180b250c1e2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:51.733051 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7adb6c-ae41-4f0f-92fa-8180b250c1e2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:51.733576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f7adb6c-ae41-4f0f-92fa-8180b250c1e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:51.772346 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4f7adb6c-ae41-4f0f-92fa-8180b250c1e2 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:51.773631 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4f7adb6c-ae41-4f0f-92fa-8180b250c1e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:51.797849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa170b7-8475-4695-95da-54dcd0bd8293 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:51.801356 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1009/2020] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 70 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:51.812298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aa170b7-8475-4695-95da-54dcd0bd8293 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:51.833834 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c42656-5d99-4dfd-8efa-f8d3637a08f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:51.834865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c42656-5d99-4dfd-8efa-f8d3637a08f7 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:51.835378 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c42656-5d99-4dfd-8efa-f8d3637a08f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:51.835667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c42656-5d99-4dfd-8efa-f8d3637a08f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:51.836513 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c42656-5d99-4dfd-8efa-f8d3637a08f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:51.836849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c42656-5d99-4dfd-8efa-f8d3637a08f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:51.840779 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c42656-5d99-4dfd-8efa-f8d3637a08f7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:51.856196 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1012/2021] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:51.861111 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7adb6c-ae41-4f0f-92fa-8180b250c1e2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:51.891758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f7adb6c-ae41-4f0f-92fa-8180b250c1e2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:51.900965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffa040-57d9-4e99-8164-31256ef4e14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:51.901692 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffa040-57d9-4e99-8164-31256ef4e14c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:51.901920 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffa040-57d9-4e99-8164-31256ef4e14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:51.902201 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ffa040-57d9-4e99-8164-31256ef4e14c admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:51.903282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffa040-57d9-4e99-8164-31256ef4e14c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:51.905251 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ffa040-57d9-4e99-8164-31256ef4e14c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:51.942682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1010/2022] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:51 2022] POST /identity/v3/projects => generated 371 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:51.946650 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c42656-5d99-4dfd-8efa-f8d3637a08f7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:51.958288 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c42656-5d99-4dfd-8efa-f8d3637a08f7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:52.025386 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1c033-0d9d-4e61-ab24-64eff54669fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:52.026187 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1c033-0d9d-4e61-ab24-64eff54669fd admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:52.026485 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1c033-0d9d-4e61-ab24-64eff54669fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:52.026774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36a1c033-0d9d-4e61-ab24-64eff54669fd admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:52.028072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1c033-0d9d-4e61-ab24-64eff54669fd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:52.030252 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36a1c033-0d9d-4e61-ab24-64eff54669fd admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:52.066054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1013/2023] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:51 2022] POST /identity/v3/users => generated 286 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:52.070774 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffa040-57d9-4e99-8164-31256ef4e14c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:52.082588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ffa040-57d9-4e99-8164-31256ef4e14c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:52.087328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4bf13-fbb5-4d43-93e0-5430c17a05c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:52.088056 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4bf13-fbb5-4d43-93e0-5430c17a05c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:52.088351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4bf13-fbb5-4d43-93e0-5430c17a05c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:52.088633 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e4bf13-fbb5-4d43-93e0-5430c17a05c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:52.089385 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4bf13-fbb5-4d43-93e0-5430c17a05c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:52.091026 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e4bf13-fbb5-4d43-93e0-5430c17a05c2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:52.102366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1011/2024] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:52.107013 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1c033-0d9d-4e61-ab24-64eff54669fd admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:52.117434 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36a1c033-0d9d-4e61-ab24-64eff54669fd admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:52.122758 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68671e44-4eae-414e-83e9-e9fa7d6367e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:52.123437 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68671e44-4eae-414e-83e9-e9fa7d6367e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:52.123729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68671e44-4eae-414e-83e9-e9fa7d6367e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:52.124049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68671e44-4eae-414e-83e9-e9fa7d6367e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:52.124787 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68671e44-4eae-414e-83e9-e9fa7d6367e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:52.126419 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68671e44-4eae-414e-83e9-e9fa7d6367e6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:52.137718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1014/2025] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:52.142680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4bf13-fbb5-4d43-93e0-5430c17a05c2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:52.153433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e4bf13-fbb5-4d43-93e0-5430c17a05c2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:52.158428 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40f0cf-ccf6-4a7c-98a7-45e9395c11db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:52.159326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40f0cf-ccf6-4a7c-98a7-45e9395c11db admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:52.159612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40f0cf-ccf6-4a7c-98a7-45e9395c11db admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:52.159957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e40f0cf-ccf6-4a7c-98a7-45e9395c11db admin admin] PATH_INFO: `/v3/projects/7ba3f6257a744db3b7065c16cb1582d7/users/928cdd6dfbe14ae5ab192ebf1a33964e/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:52.175847 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e40f0cf-ccf6-4a7c-98a7-45e9395c11db admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ba3f6257a744db3b7065c16cb1582d7, user_id=928cdd6dfbe14ae5ab192ebf1a33964e, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:52.178592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e40f0cf-ccf6-4a7c-98a7-45e9395c11db admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:52.193383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1012/2026] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:52 2022] PUT /identity/v3/projects/7ba3f6257a744db3b7065c16cb1582d7/users/928cdd6dfbe14ae5ab192ebf1a33964e/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:52.203973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b03c8e-883a-4c1f-bea2-b378ec5687eb None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:52.204270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b03c8e-883a-4c1f-bea2-b378ec5687eb None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:52.204552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b03c8e-883a-4c1f-bea2-b378ec5687eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:52.233017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-72b03c8e-883a-4c1f-bea2-b378ec5687eb None None] MFA Rules not processed for user `928cdd6dfbe14ae5ab192ebf1a33964e`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:52.234313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-72b03c8e-883a-4c1f-bea2-b378ec5687eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:52.281064 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:52.315568 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1015/2027] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:52.328020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40f0cf-ccf6-4a7c-98a7-45e9395c11db admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:52.330693 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1e40f0cf-ccf6-4a7c-98a7-45e9395c11db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:52.356311 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b03c8e-883a-4c1f-bea2-b378ec5687eb None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:52.358077 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-72b03c8e-883a-4c1f-bea2-b378ec5687eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:52.426091 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e40f0cf-ccf6-4a7c-98a7-45e9395c11db admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:52.430352 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e08327-17c7-46d9-8db9-a2e20ad3e022 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:52.431713 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e08327-17c7-46d9-8db9-a2e20ad3e022 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:52.432103 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e08327-17c7-46d9-8db9-a2e20ad3e022 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:52.432464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e08327-17c7-46d9-8db9-a2e20ad3e022 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:52.442576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e08327-17c7-46d9-8db9-a2e20ad3e022 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:52.444365 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e08327-17c7-46d9-8db9-a2e20ad3e022 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:52.449047 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b03c8e-883a-4c1f-bea2-b378ec5687eb None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:52.454190 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb900031-0442-446c-b0c0-a400a0f8e004 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:52.455291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb900031-0442-446c-b0c0-a400a0f8e004 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:52.455761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb900031-0442-446c-b0c0-a400a0f8e004 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:52.455919 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb900031-0442-446c-b0c0-a400a0f8e004 admin admin] PATH_INFO: `/v3/users/6b91cef758a0482b8469b05a0d0034f5` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:52.456481 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1013/2028] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:52 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:52.457266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb900031-0442-446c-b0c0-a400a0f8e004 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b91cef758a0482b8469b05a0d0034f5)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:52.459369 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb900031-0442-446c-b0c0-a400a0f8e004 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:52.475019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e08327-17c7-46d9-8db9-a2e20ad3e022 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:52.485579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e08327-17c7-46d9-8db9-a2e20ad3e022 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:52.492632 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5286bcfa-48c2-4eed-a69f-4a180e2308f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:52.493516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286bcfa-48c2-4eed-a69f-4a180e2308f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:52.493807 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286bcfa-48c2-4eed-a69f-4a180e2308f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:52.494087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5286bcfa-48c2-4eed-a69f-4a180e2308f6 admin admin] PATH_INFO: `/v3/users/928cdd6dfbe14ae5ab192ebf1a33964e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:52.495315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286bcfa-48c2-4eed-a69f-4a180e2308f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=928cdd6dfbe14ae5ab192ebf1a33964e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:52.497266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5286bcfa-48c2-4eed-a69f-4a180e2308f6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:52.513179 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-eb900031-0442-446c-b0c0-a400a0f8e004 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b91cef758a0482b8469b05a0d0034f5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:52.513956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-eb900031-0442-446c-b0c0-a400a0f8e004 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b91cef758a0482b8469b05a0d0034f5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:52.529941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-eb900031-0442-446c-b0c0-a400a0f8e004 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b91cef758a0482b8469b05a0d0034f5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:52.539125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:52.539382 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-eb900031-0442-446c-b0c0-a400a0f8e004 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b91cef758a0482b8469b05a0d0034f5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:52.540125 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-eb900031-0442-446c-b0c0-a400a0f8e004 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b91cef758a0482b8469b05a0d0034f5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:52.556490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1016/2029] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:52 2022] DELETE /identity/v3/users/6b91cef758a0482b8469b05a0d0034f5 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:52.561808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a94c0-7ec6-4307-a886-810f77369436 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:52.562144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a94c0-7ec6-4307-a886-810f77369436 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:52.562592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41a94c0-7ec6-4307-a886-810f77369436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:52.578608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5286bcfa-48c2-4eed-a69f-4a180e2308f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '928cdd6dfbe14ae5ab192ebf1a33964e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:52.582350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c41a94c0-7ec6-4307-a886-810f77369436 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:52.583741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c41a94c0-7ec6-4307-a886-810f77369436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:52.596858 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5286bcfa-48c2-4eed-a69f-4a180e2308f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '928cdd6dfbe14ae5ab192ebf1a33964e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:52.609471 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5286bcfa-48c2-4eed-a69f-4a180e2308f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '928cdd6dfbe14ae5ab192ebf1a33964e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:52.610324 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5286bcfa-48c2-4eed-a69f-4a180e2308f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '928cdd6dfbe14ae5ab192ebf1a33964e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:52.620632 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5286bcfa-48c2-4eed-a69f-4a180e2308f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '928cdd6dfbe14ae5ab192ebf1a33964e'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:52.627336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1014/2030] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:52 2022] DELETE /identity/v3/users/928cdd6dfbe14ae5ab192ebf1a33964e => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:52.631932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0335caf-d79c-4d16-a836-d9efb763a767 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:52.632169 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0335caf-d79c-4d16-a836-d9efb763a767 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:52.632622 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0335caf-d79c-4d16-a836-d9efb763a767 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:52.653875 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f0335caf-d79c-4d16-a836-d9efb763a767 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:52.655055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f0335caf-d79c-4d16-a836-d9efb763a767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:52.666164 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1017/2031] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:08:52.678132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a94c0-7ec6-4307-a886-810f77369436 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:52.680045 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c41a94c0-7ec6-4307-a886-810f77369436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:52.685498 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:52.735921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1015/2032] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:52.745511 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0335caf-d79c-4d16-a836-d9efb763a767 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:52.747203 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f0335caf-d79c-4d16-a836-d9efb763a767 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:52.761963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41a94c0-7ec6-4307-a886-810f77369436 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:52.847098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c303dcd-fb18-429a-a5a6-02df73073494 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:52.847988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c303dcd-fb18-429a-a5a6-02df73073494 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:52.848382 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c303dcd-fb18-429a-a5a6-02df73073494 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:52.849055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c303dcd-fb18-429a-a5a6-02df73073494 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:52.864410 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0335caf-d79c-4d16-a836-d9efb763a767 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:52.871044 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc79cb-3652-46f6-821f-c14ef5ee06f1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:52.871789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc79cb-3652-46f6-821f-c14ef5ee06f1 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:52.872170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc79cb-3652-46f6-821f-c14ef5ee06f1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:52.872484 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56cc79cb-3652-46f6-821f-c14ef5ee06f1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:52.873995 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c303dcd-fb18-429a-a5a6-02df73073494 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:52.875473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c303dcd-fb18-429a-a5a6-02df73073494 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:52.886003 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc79cb-3652-46f6-821f-c14ef5ee06f1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:52.887686 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56cc79cb-3652-46f6-821f-c14ef5ee06f1 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:52.898294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1018/2033] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 223 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:08:52.900539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1016/2034] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:08:53.369018 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c303dcd-fb18-429a-a5a6-02df73073494 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:53.370498 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7c303dcd-fb18-429a-a5a6-02df73073494 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:53.386504 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c303dcd-fb18-429a-a5a6-02df73073494 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:53.390021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc79cb-3652-46f6-821f-c14ef5ee06f1 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:53.394205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bb028bf-e6b3-43dc-bea7-96fbd2c6f188 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:53.395449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb028bf-e6b3-43dc-bea7-96fbd2c6f188 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:53.396089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb028bf-e6b3-43dc-bea7-96fbd2c6f188 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:53.396577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bb028bf-e6b3-43dc-bea7-96fbd2c6f188 admin admin] PATH_INFO: `/v3/projects/fa2a5dfc856e4c2e8d120bb546bb8588` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:53.396751 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-56cc79cb-3652-46f6-821f-c14ef5ee06f1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:53.398407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb028bf-e6b3-43dc-bea7-96fbd2c6f188 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa2a5dfc856e4c2e8d120bb546bb8588)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:53.403072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bb028bf-e6b3-43dc-bea7-96fbd2c6f188 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:53.433639 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56cc79cb-3652-46f6-821f-c14ef5ee06f1 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:53.441188 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f21badd0-3a51-4f17-97a8-bd30b02d4e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:53.442117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21badd0-3a51-4f17-97a8-bd30b02d4e6a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:53.442468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21badd0-3a51-4f17-97a8-bd30b02d4e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:53.442804 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21badd0-3a51-4f17-97a8-bd30b02d4e6a admin admin] PATH_INFO: `/v3/projects/7ba3f6257a744db3b7065c16cb1582d7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:53.444083 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21badd0-3a51-4f17-97a8-bd30b02d4e6a admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ba3f6257a744db3b7065c16cb1582d7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:53.446119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f21badd0-3a51-4f17-97a8-bd30b02d4e6a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:53.472674 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bb028bf-e6b3-43dc-bea7-96fbd2c6f188 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa2a5dfc856e4c2e8d120bb546bb8588', 'request_id': 'req-0bb028bf-e6b3-43dc-bea7-96fbd2c6f188'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:53.484891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bb028bf-e6b3-43dc-bea7-96fbd2c6f188 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa2a5dfc856e4c2e8d120bb546bb8588', 'request_id': 'req-0bb028bf-e6b3-43dc-bea7-96fbd2c6f188'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:53.498227 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bb028bf-e6b3-43dc-bea7-96fbd2c6f188 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa2a5dfc856e4c2e8d120bb546bb8588', 'request_id': 'req-0bb028bf-e6b3-43dc-bea7-96fbd2c6f188'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:53.500991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:53.508854 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f21badd0-3a51-4f17-97a8-bd30b02d4e6a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ba3f6257a744db3b7065c16cb1582d7', 'request_id': 'req-f21badd0-3a51-4f17-97a8-bd30b02d4e6a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:53.513397 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bb028bf-e6b3-43dc-bea7-96fbd2c6f188 admin admin] The token cache is being invalidate because project fa2a5dfc856e4c2e8d120bb546bb8588 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:53.513855 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bb028bf-e6b3-43dc-bea7-96fbd2c6f188 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:53.514396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bb028bf-e6b3-43dc-bea7-96fbd2c6f188 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:53.516185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1019/2035] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:53 2022] DELETE /identity/v3/projects/fa2a5dfc856e4c2e8d120bb546bb8588 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:53.518989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f21badd0-3a51-4f17-97a8-bd30b02d4e6a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ba3f6257a744db3b7065c16cb1582d7', 'request_id': 'req-f21badd0-3a51-4f17-97a8-bd30b02d4e6a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:53.521347 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2275881-9ca0-4c13-b1f6-7c155cf17dd7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:53.521641 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2275881-9ca0-4c13-b1f6-7c155cf17dd7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:53.521929 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2275881-9ca0-4c13-b1f6-7c155cf17dd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:53.525986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f21badd0-3a51-4f17-97a8-bd30b02d4e6a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ba3f6257a744db3b7065c16cb1582d7', 'request_id': 'req-f21badd0-3a51-4f17-97a8-bd30b02d4e6a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:53.538825 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f21badd0-3a51-4f17-97a8-bd30b02d4e6a admin admin] The token cache is being invalidate because project 7ba3f6257a744db3b7065c16cb1582d7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:53.539201 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f21badd0-3a51-4f17-97a8-bd30b02d4e6a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:53.540199 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f21badd0-3a51-4f17-97a8-bd30b02d4e6a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:53.542053 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1017/2036] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:53 2022] DELETE /identity/v3/projects/7ba3f6257a744db3b7065c16cb1582d7 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:53.545259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c2275881-9ca0-4c13-b1f6-7c155cf17dd7 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:53.546541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ad587-f149-4087-9240-342e0038792a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:53.546719 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c2275881-9ca0-4c13-b1f6-7c155cf17dd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:53.546832 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ad587-f149-4087-9240-342e0038792a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:53.547123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d1ad587-f149-4087-9240-342e0038792a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:53.583736 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9d1ad587-f149-4087-9240-342e0038792a None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:53.585628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9d1ad587-f149-4087-9240-342e0038792a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:53.598696 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:53.620003 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1020/2037] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:53.627723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25242a14-1cf3-4c9e-955d-e9cd1ed41b60 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:53.628046 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25242a14-1cf3-4c9e-955d-e9cd1ed41b60 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:53.628560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25242a14-1cf3-4c9e-955d-e9cd1ed41b60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:53.646479 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-25242a14-1cf3-4c9e-955d-e9cd1ed41b60 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:53.646934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1018/2038] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:08:53.647804 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-25242a14-1cf3-4c9e-955d-e9cd1ed41b60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:53.654907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9d1c1-0566-4e23-adb4-ac8528eb710c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:53.655211 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9d1c1-0566-4e23-adb4-ac8528eb710c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:53.655501 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e9d1c1-0566-4e23-adb4-ac8528eb710c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:53.656050 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1021/2039] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:08:53.660412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25242a14-1cf3-4c9e-955d-e9cd1ed41b60 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:53.672377 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25242a14-1cf3-4c9e-955d-e9cd1ed41b60 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:53.675465 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-05e9d1c1-0566-4e23-adb4-ac8528eb710c None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:53.676983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-05e9d1c1-0566-4e23-adb4-ac8528eb710c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:53.678033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bc961-311c-4fb5-8e3a-e6f2c12082b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:53.678726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bc961-311c-4fb5-8e3a-e6f2c12082b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:53.679016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bc961-311c-4fb5-8e3a-e6f2c12082b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:53.679415 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-145bc961-311c-4fb5-8e3a-e6f2c12082b7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:53.680191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bc961-311c-4fb5-8e3a-e6f2c12082b7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:53.680523 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bc961-311c-4fb5-8e3a-e6f2c12082b7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:53.682181 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-145bc961-311c-4fb5-8e3a-e6f2c12082b7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:53.685416 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1019/2040] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:08:53.689674 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9d1c1-0566-4e23-adb4-ac8528eb710c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:53.695302 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1022/2041] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:53.699071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e9d1c1-0566-4e23-adb4-ac8528eb710c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:53.699640 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bc961-311c-4fb5-8e3a-e6f2c12082b7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:53.703951 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb91803-a253-46cb-986c-b8ef12e3dbdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:53.704622 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb91803-a253-46cb-986c-b8ef12e3dbdf admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:53.704917 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb91803-a253-46cb-986c-b8ef12e3dbdf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:53.705200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb91803-a253-46cb-986c-b8ef12e3dbdf admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:53.705937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb91803-a253-46cb-986c-b8ef12e3dbdf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:53.706260 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb91803-a253-46cb-986c-b8ef12e3dbdf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:53.707958 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb91803-a253-46cb-986c-b8ef12e3dbdf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:53.709537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-145bc961-311c-4fb5-8e3a-e6f2c12082b7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:53.714815 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d10e3a-24ad-415b-b4bf-4d4926602e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:53.715481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d10e3a-24ad-415b-b4bf-4d4926602e1b admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:53.715773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d10e3a-24ad-415b-b4bf-4d4926602e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:53.716099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d10e3a-24ad-415b-b4bf-4d4926602e1b admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:53.717359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d10e3a-24ad-415b-b4bf-4d4926602e1b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:53.719989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d10e3a-24ad-415b-b4bf-4d4926602e1b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:53.720449 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1020/2042] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:08:53.726436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb91803-a253-46cb-986c-b8ef12e3dbdf admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:53.736964 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb91803-a253-46cb-986c-b8ef12e3dbdf admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:53.743041 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b5e14-f424-4a15-af5e-65d025268493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:53.743720 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b5e14-f424-4a15-af5e-65d025268493 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:53.744056 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b5e14-f424-4a15-af5e-65d025268493 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:53.744345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995b5e14-f424-4a15-af5e-65d025268493 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:53.745099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b5e14-f424-4a15-af5e-65d025268493 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:53.747064 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-995b5e14-f424-4a15-af5e-65d025268493 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:53.749324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1023/2043] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:53 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:53.753348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d10e3a-24ad-415b-b4bf-4d4926602e1b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:53.777858 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1021/2044] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:53 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:53.781800 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b5e14-f424-4a15-af5e-65d025268493 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:53.817588 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d10e3a-24ad-415b-b4bf-4d4926602e1b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:53.844694 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995b5e14-f424-4a15-af5e-65d025268493 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:53.849355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43ccea-0363-4378-8201-c53c02e047f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:53.850042 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43ccea-0363-4378-8201-c53c02e047f0 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:53.850337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43ccea-0363-4378-8201-c53c02e047f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:53.850654 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa43ccea-0363-4378-8201-c53c02e047f0 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:53.851359 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43ccea-0363-4378-8201-c53c02e047f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:53.853346 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa43ccea-0363-4378-8201-c53c02e047f0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:53.874732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09a675-40e2-4172-a9f2-de935dc1300d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:53.878272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1022/2045] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:53 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:53.878399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09a675-40e2-4172-a9f2-de935dc1300d admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:53.878399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09a675-40e2-4172-a9f2-de935dc1300d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:53.878399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb09a675-40e2-4172-a9f2-de935dc1300d admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:53.878399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09a675-40e2-4172-a9f2-de935dc1300d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:53.878968 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb09a675-40e2-4172-a9f2-de935dc1300d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:53.879298 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43ccea-0363-4378-8201-c53c02e047f0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:53.889979 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa43ccea-0363-4378-8201-c53c02e047f0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:53.895309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c6f05-d524-4401-8470-f82fdc542c94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:53.896065 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c6f05-d524-4401-8470-f82fdc542c94 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:53.896358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c6f05-d524-4401-8470-f82fdc542c94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:53.896642 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9c6f05-d524-4401-8470-f82fdc542c94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:53.897378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c6f05-d524-4401-8470-f82fdc542c94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:53.899507 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9c6f05-d524-4401-8470-f82fdc542c94 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:53.900248 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1024/2046] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:53 2022] POST /identity/v3/users => generated 286 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:53.904421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09a675-40e2-4172-a9f2-de935dc1300d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:53.911046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1023/2047] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:53.914632 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c6f05-d524-4401-8470-f82fdc542c94 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:53.916954 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb09a675-40e2-4172-a9f2-de935dc1300d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:53.922247 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02694109-95d5-4b04-b0db-c70ce45cb9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:53.922928 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02694109-95d5-4b04-b0db-c70ce45cb9e3 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:53.923516 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02694109-95d5-4b04-b0db-c70ce45cb9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:53.923516 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02694109-95d5-4b04-b0db-c70ce45cb9e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:53.924250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02694109-95d5-4b04-b0db-c70ce45cb9e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:53.925863 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02694109-95d5-4b04-b0db-c70ce45cb9e3 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:53.928716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9c6f05-d524-4401-8470-f82fdc542c94 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:53.937607 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f420c44-52b6-4fd8-b1ac-132d6e371f35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:53.938315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f420c44-52b6-4fd8-b1ac-132d6e371f35 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:53.938654 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f420c44-52b6-4fd8-b1ac-132d6e371f35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:53.938941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f420c44-52b6-4fd8-b1ac-132d6e371f35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:53.942091 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f420c44-52b6-4fd8-b1ac-132d6e371f35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:53.945868 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1025/2048] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:53.948023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f420c44-52b6-4fd8-b1ac-132d6e371f35 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:53.948023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:53.948497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02694109-95d5-4b04-b0db-c70ce45cb9e3 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:53.963991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1024/2049] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:53.965621 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02694109-95d5-4b04-b0db-c70ce45cb9e3 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:53.966923 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f420c44-52b6-4fd8-b1ac-132d6e371f35 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:53.971446 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c0628-decc-4cc2-8c85-84fc93823c30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:53.971877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c0628-decc-4cc2-8c85-84fc93823c30 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:53.972111 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c0628-decc-4cc2-8c85-84fc93823c30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:53.972398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a15c0628-decc-4cc2-8c85-84fc93823c30 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:53.973144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c0628-decc-4cc2-8c85-84fc93823c30 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:53.974855 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a15c0628-decc-4cc2-8c85-84fc93823c30 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:53.984749 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f420c44-52b6-4fd8-b1ac-132d6e371f35 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:53.990264 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1026/2050] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:53 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:53.991662 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c0628-decc-4cc2-8c85-84fc93823c30 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:53.994244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dffdb2f-bc60-425a-942f-99e282415643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:53.995368 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffdb2f-bc60-425a-942f-99e282415643 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:53.995954 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffdb2f-bc60-425a-942f-99e282415643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:53.996424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dffdb2f-bc60-425a-942f-99e282415643 admin admin] PATH_INFO: `/v3/projects/5a3449bb5bef4c469973ee9e65d5a134/users/306a1138981b400da68cf31e03afaaca/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:54.003278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a15c0628-decc-4cc2-8c85-84fc93823c30 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:54.009063 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a1f1a14-2d93-4c55-82bf-e55e90a0873a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:54.010128 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f1a14-2d93-4c55-82bf-e55e90a0873a admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:54.010411 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f1a14-2d93-4c55-82bf-e55e90a0873a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:54.010688 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a1f1a14-2d93-4c55-82bf-e55e90a0873a admin admin] PATH_INFO: `/v3/projects/57f9464c88474b21a98f614f3d7a911c/users/b7db815d31cf4c54845f5f8c9dcb1b17/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:54.017544 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffdb2f-bc60-425a-942f-99e282415643 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5a3449bb5bef4c469973ee9e65d5a134, user_id=306a1138981b400da68cf31e03afaaca, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:54.020500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dffdb2f-bc60-425a-942f-99e282415643 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:54.048414 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1025/2051] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:53 2022] PUT /identity/v3/projects/5a3449bb5bef4c469973ee9e65d5a134/users/306a1138981b400da68cf31e03afaaca/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:54.056516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7491d-4378-409d-9688-2ddd0cefbaf2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:54.056877 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7491d-4378-409d-9688-2ddd0cefbaf2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:54.057098 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b7491d-4378-409d-9688-2ddd0cefbaf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:54.061039 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f1a14-2d93-4c55-82bf-e55e90a0873a admin admin] RBAC: Authorizing `identity:create_grant(project_id=57f9464c88474b21a98f614f3d7a911c, user_id=b7db815d31cf4c54845f5f8c9dcb1b17, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:54.063803 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a1f1a14-2d93-4c55-82bf-e55e90a0873a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:54.066428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:54.087987 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1027/2052] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:53 2022] PUT /identity/v3/projects/57f9464c88474b21a98f614f3d7a911c/users/b7db815d31cf4c54845f5f8c9dcb1b17/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 93 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:54.095444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1484285-86e2-4d54-a4fa-c51b46d2a04f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:54.095444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1484285-86e2-4d54-a4fa-c51b46d2a04f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:54.095444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1484285-86e2-4d54-a4fa-c51b46d2a04f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:54.107433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-51b7491d-4378-409d-9688-2ddd0cefbaf2 None None] MFA Rules not processed for user `306a1138981b400da68cf31e03afaaca`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:54.108936 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-51b7491d-4378-409d-9688-2ddd0cefbaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:54.148636 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f1484285-86e2-4d54-a4fa-c51b46d2a04f None None] MFA Rules not processed for user `b7db815d31cf4c54845f5f8c9dcb1b17`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:54.148636 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f1484285-86e2-4d54-a4fa-c51b46d2a04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:54.219985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1026/2053] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:54 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:08:54.233209 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7491d-4378-409d-9688-2ddd0cefbaf2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:54.235317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-51b7491d-4378-409d-9688-2ddd0cefbaf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:54.240465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1028/2054] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:08:54.252276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1484285-86e2-4d54-a4fa-c51b46d2a04f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:54.254278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f1484285-86e2-4d54-a4fa-c51b46d2a04f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:54.322574 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b7491d-4378-409d-9688-2ddd0cefbaf2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:54.330154 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1484285-86e2-4d54-a4fa-c51b46d2a04f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:54.330392 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444ba60c-a991-4a73-81ce-35dfe3f994e4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:54.331193 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ba60c-a991-4a73-81ce-35dfe3f994e4 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:54.331491 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ba60c-a991-4a73-81ce-35dfe3f994e4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:54.331871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444ba60c-a991-4a73-81ce-35dfe3f994e4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:54.339369 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f6bce4-bdac-42d4-93d0-95f157697659 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '196dc406a95444c39babc50d59f9eb6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:54.340403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6bce4-bdac-42d4-93d0-95f157697659 service glance] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:54.340742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6bce4-bdac-42d4-93d0-95f157697659 service glance] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:54.341166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f6bce4-bdac-42d4-93d0-95f157697659 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:54.341716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ba60c-a991-4a73-81ce-35dfe3f994e4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:54.343320 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444ba60c-a991-4a73-81ce-35dfe3f994e4 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:54.353345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1027/2055] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:54 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:54.355544 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6bce4-bdac-42d4-93d0-95f157697659 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:54.357047 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f6bce4-bdac-42d4-93d0-95f157697659 service glance] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:54.366708 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1029/2056] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:08:56.365909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fef550-3dc5-4471-af8c-26f50f3d331a None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:56.366506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fef550-3dc5-4471-af8c-26f50f3d331a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:56.366506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3fef550-3dc5-4471-af8c-26f50f3d331a None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:56.368051 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1028/2057] 10.222.0.85 () {54 vars in 921 bytes} [Wed May 25 16:08:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:08:56.382111 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621570ec-9d48-42b8-8c86-1103c621bab4 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:56.382671 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621570ec-9d48-42b8-8c86-1103c621bab4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:56.382671 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621570ec-9d48-42b8-8c86-1103c621bab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:56.428205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-621570ec-9d48-42b8-8c86-1103c621bab4 None None] MFA Rules not processed for user `82b738f1b8c64590955a640c461b4ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:56.430137 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-621570ec-9d48-42b8-8c86-1103c621bab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:56.466926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fef550-3dc5-4471-af8c-26f50f3d331a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:56.472912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e3fef550-3dc5-4471-af8c-26f50f3d331a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:56.523973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1030/2058] 10.222.0.85 () {58 vars in 1018 bytes} [Wed May 25 16:08:56 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:56.535679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621570ec-9d48-42b8-8c86-1103c621bab4 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:56.537280 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-621570ec-9d48-42b8-8c86-1103c621bab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:56.552867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621570ec-9d48-42b8-8c86-1103c621bab4 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:56.558655 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafb652-97b5-408c-82cf-054378431c34 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b738f1b8c64590955a640c461b4ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:56.559392 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafb652-97b5-408c-82cf-054378431c34 service placement] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:56.559687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafb652-97b5-408c-82cf-054378431c34 service placement] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:56.560107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eafb652-97b5-408c-82cf-054378431c34 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:56.561837 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3fef550-3dc5-4471-af8c-26f50f3d331a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:56.566785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65b50ef1-76ad-44eb-8007-2a7966f64c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:56.568419 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b50ef1-76ad-44eb-8007-2a7966f64c19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:56.568419 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b50ef1-76ad-44eb-8007-2a7966f64c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:56.568662 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65b50ef1-76ad-44eb-8007-2a7966f64c19 admin admin] PATH_INFO: `/v3/users/1d4d043b13da4b17830481f4a8da9b79` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:56.570236 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b50ef1-76ad-44eb-8007-2a7966f64c19 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1d4d043b13da4b17830481f4a8da9b79)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:56.571259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafb652-97b5-408c-82cf-054378431c34 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:56.572373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65b50ef1-76ad-44eb-8007-2a7966f64c19 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:56.572877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eafb652-97b5-408c-82cf-054378431c34 service placement] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:56.578122 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:56.582898 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1031/2059] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:56 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:08:56.630673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-65b50ef1-76ad-44eb-8007-2a7966f64c19 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1d4d043b13da4b17830481f4a8da9b79'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:56.642900 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-65b50ef1-76ad-44eb-8007-2a7966f64c19 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1d4d043b13da4b17830481f4a8da9b79'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:56.651476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-65b50ef1-76ad-44eb-8007-2a7966f64c19 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1d4d043b13da4b17830481f4a8da9b79'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:56.651958 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-65b50ef1-76ad-44eb-8007-2a7966f64c19 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1d4d043b13da4b17830481f4a8da9b79'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:56.660806 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-65b50ef1-76ad-44eb-8007-2a7966f64c19 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1d4d043b13da4b17830481f4a8da9b79'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:56.666990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1029/2060] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:56 2022] DELETE /identity/v3/users/1d4d043b13da4b17830481f4a8da9b79 => generated 0 bytes in 201 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:56.842435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafb652-97b5-408c-82cf-054378431c34 service placement] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:56.855726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eafb652-97b5-408c-82cf-054378431c34 service placement] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:56.862411 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:INSERT=1,keystone:SELECT=99 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:56.924087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928513c7-036a-4c41-bf30-7779068b6fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:56.925017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928513c7-036a-4c41-bf30-7779068b6fa8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:56.925307 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928513c7-036a-4c41-bf30-7779068b6fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:56.925618 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928513c7-036a-4c41-bf30-7779068b6fa8 admin admin] PATH_INFO: `/v3/projects/1f6d103cb389478f94ce6afe687c0e28` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:56.926835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928513c7-036a-4c41-bf30-7779068b6fa8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f6d103cb389478f94ce6afe687c0e28)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:56.928839 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928513c7-036a-4c41-bf30-7779068b6fa8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:56.986451 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-928513c7-036a-4c41-bf30-7779068b6fa8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f6d103cb389478f94ce6afe687c0e28', 'request_id': 'req-928513c7-036a-4c41-bf30-7779068b6fa8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:56.991994 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-928513c7-036a-4c41-bf30-7779068b6fa8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f6d103cb389478f94ce6afe687c0e28', 'request_id': 'req-928513c7-036a-4c41-bf30-7779068b6fa8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:57.001590 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-928513c7-036a-4c41-bf30-7779068b6fa8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f6d103cb389478f94ce6afe687c0e28', 'request_id': 'req-928513c7-036a-4c41-bf30-7779068b6fa8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:57.013346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-928513c7-036a-4c41-bf30-7779068b6fa8 admin admin] The token cache is being invalidate because project 1f6d103cb389478f94ce6afe687c0e28 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:57.013642 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-928513c7-036a-4c41-bf30-7779068b6fa8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:57.014371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-928513c7-036a-4c41-bf30-7779068b6fa8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:57.016029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1032/2061] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:56 2022] DELETE /identity/v3/projects/1f6d103cb389478f94ce6afe687c0e28 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:57.021308 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7851055-97f4-40e2-a686-80ab471337c4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:57.021607 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7851055-97f4-40e2-a686-80ab471337c4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:57.021892 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7851055-97f4-40e2-a686-80ab471337c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:57.039917 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a7851055-97f4-40e2-a686-80ab471337c4 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:57.041134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a7851055-97f4-40e2-a686-80ab471337c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:57.101597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1030/2062] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:57.109822 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedc6ad-fd9e-46d0-bdce-6cdfa8b9ec5a None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:57.110242 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedc6ad-fd9e-46d0-bdce-6cdfa8b9ec5a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:57.110752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faedc6ad-fd9e-46d0-bdce-6cdfa8b9ec5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:57.130913 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-faedc6ad-fd9e-46d0-bdce-6cdfa8b9ec5a None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:57.132180 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-faedc6ad-fd9e-46d0-bdce-6cdfa8b9ec5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:57.141756 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1033/2063] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:57.146108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7851055-97f4-40e2-a686-80ab471337c4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:57.156559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7851055-97f4-40e2-a686-80ab471337c4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:57.161636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c7660-d9ca-4c8f-82e6-db3b6294d140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:57.162247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c7660-d9ca-4c8f-82e6-db3b6294d140 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:57.162540 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c7660-d9ca-4c8f-82e6-db3b6294d140 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:57.162843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a9c7660-d9ca-4c8f-82e6-db3b6294d140 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:57.163581 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c7660-d9ca-4c8f-82e6-db3b6294d140 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:08:57.163928 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c7660-d9ca-4c8f-82e6-db3b6294d140 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:57.165552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a9c7660-d9ca-4c8f-82e6-db3b6294d140 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:57.177623 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1031/2064] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:08:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:08:57.181765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faedc6ad-fd9e-46d0-bdce-6cdfa8b9ec5a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:57.191169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faedc6ad-fd9e-46d0-bdce-6cdfa8b9ec5a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:57.196091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285e961-06a3-42b0-b52d-e2940952bf41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:57.196751 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285e961-06a3-42b0-b52d-e2940952bf41 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:57.197037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285e961-06a3-42b0-b52d-e2940952bf41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:57.197314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d285e961-06a3-42b0-b52d-e2940952bf41 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:57.198020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285e961-06a3-42b0-b52d-e2940952bf41 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:57.200003 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d285e961-06a3-42b0-b52d-e2940952bf41 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:57.229869 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1034/2065] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:57 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:57.234116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c7660-d9ca-4c8f-82e6-db3b6294d140 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:57.246468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a9c7660-d9ca-4c8f-82e6-db3b6294d140 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:57.307210 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce216701-0262-4ecc-944b-cc63a5fb2411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:57.307934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce216701-0262-4ecc-944b-cc63a5fb2411 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:57.308242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce216701-0262-4ecc-944b-cc63a5fb2411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:57.308527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce216701-0262-4ecc-944b-cc63a5fb2411 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:57.309248 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce216701-0262-4ecc-944b-cc63a5fb2411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:57.311197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce216701-0262-4ecc-944b-cc63a5fb2411 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:57.329205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:57.336318 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1032/2066] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:57 2022] POST /identity/v3/users => generated 283 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:57.339514 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285e961-06a3-42b0-b52d-e2940952bf41 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:57.349926 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d285e961-06a3-42b0-b52d-e2940952bf41 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:57.354922 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde56cb-c06a-494b-b155-29b532b7b76a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:57.355672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde56cb-c06a-494b-b155-29b532b7b76a admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:57.356046 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde56cb-c06a-494b-b155-29b532b7b76a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:57.356454 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dde56cb-c06a-494b-b155-29b532b7b76a admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:57.357337 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde56cb-c06a-494b-b155-29b532b7b76a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:57.359061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dde56cb-c06a-494b-b155-29b532b7b76a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:57.372009 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1035/2067] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:57.375079 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce216701-0262-4ecc-944b-cc63a5fb2411 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:57.385512 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce216701-0262-4ecc-944b-cc63a5fb2411 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:57.390913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aee356-7d3e-4086-9ef8-8cc66ae929af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:57.391606 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aee356-7d3e-4086-9ef8-8cc66ae929af admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:57.391931 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aee356-7d3e-4086-9ef8-8cc66ae929af admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:57.392214 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aee356-7d3e-4086-9ef8-8cc66ae929af admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:57.392991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aee356-7d3e-4086-9ef8-8cc66ae929af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:57.394568 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aee356-7d3e-4086-9ef8-8cc66ae929af admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:57.406026 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1033/2068] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:57.410518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde56cb-c06a-494b-b155-29b532b7b76a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:57.421060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dde56cb-c06a-494b-b155-29b532b7b76a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:57.426819 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d2b3a-d38e-4932-af80-35924b1e3265 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:57.427721 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d2b3a-d38e-4932-af80-35924b1e3265 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:57.428047 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d2b3a-d38e-4932-af80-35924b1e3265 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:57.428330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c81d2b3a-d38e-4932-af80-35924b1e3265 admin admin] PATH_INFO: `/v3/projects/4a8e487eb8b04c3c9a3466aaaa8fb47d/users/8344101ea9c04d2292a19ad36461ba9c/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:57.444137 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d2b3a-d38e-4932-af80-35924b1e3265 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4a8e487eb8b04c3c9a3466aaaa8fb47d, user_id=8344101ea9c04d2292a19ad36461ba9c, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:57.446807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c81d2b3a-d38e-4932-af80-35924b1e3265 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:57.466820 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1036/2069] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:57 2022] PUT /identity/v3/projects/4a8e487eb8b04c3c9a3466aaaa8fb47d/users/8344101ea9c04d2292a19ad36461ba9c/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:57.480192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f559a36-a224-4140-bbbc-a81f77339793 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:57.480491 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f559a36-a224-4140-bbbc-a81f77339793 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:57.480768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f559a36-a224-4140-bbbc-a81f77339793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:57.498269 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d2b3a-d38e-4932-af80-35924b1e3265 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:57.500318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c81d2b3a-d38e-4932-af80-35924b1e3265 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:57.519908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7f559a36-a224-4140-bbbc-a81f77339793 None None] MFA Rules not processed for user `8344101ea9c04d2292a19ad36461ba9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:57.522045 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7f559a36-a224-4140-bbbc-a81f77339793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:57.547166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:57.577016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c81d2b3a-d38e-4932-af80-35924b1e3265 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:57.582062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d458db27-80b6-42fd-89e3-735a64755210 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:57.582981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458db27-80b6-42fd-89e3-735a64755210 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:57.583345 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458db27-80b6-42fd-89e3-735a64755210 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:57.583639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d458db27-80b6-42fd-89e3-735a64755210 admin admin] PATH_INFO: `/v3/users/306a1138981b400da68cf31e03afaaca` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:57.584967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458db27-80b6-42fd-89e3-735a64755210 admin admin] RBAC: Authorizing `identity:delete_user(user_id=306a1138981b400da68cf31e03afaaca)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:57.586898 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d458db27-80b6-42fd-89e3-735a64755210 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:57.594137 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1034/2070] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:57 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:57.597917 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f559a36-a224-4140-bbbc-a81f77339793 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:57.609055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f559a36-a224-4140-bbbc-a81f77339793 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:57.613714 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed109382-99eb-42b5-a2cd-c91e58364029 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:57.614419 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed109382-99eb-42b5-a2cd-c91e58364029 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:57.614733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed109382-99eb-42b5-a2cd-c91e58364029 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:57.615021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed109382-99eb-42b5-a2cd-c91e58364029 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:57.616062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed109382-99eb-42b5-a2cd-c91e58364029 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:57.618544 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed109382-99eb-42b5-a2cd-c91e58364029 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:57.641342 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d458db27-80b6-42fd-89e3-735a64755210 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '306a1138981b400da68cf31e03afaaca'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:57.642262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d458db27-80b6-42fd-89e3-735a64755210 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '306a1138981b400da68cf31e03afaaca'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:57.649925 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1035/2071] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:08:57 2022] POST /identity/v3/projects => generated 365 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:08:57.656327 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed109382-99eb-42b5-a2cd-c91e58364029 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:57.664520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d458db27-80b6-42fd-89e3-735a64755210 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '306a1138981b400da68cf31e03afaaca'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:57.675415 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d458db27-80b6-42fd-89e3-735a64755210 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '306a1138981b400da68cf31e03afaaca'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:57.676059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d458db27-80b6-42fd-89e3-735a64755210 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '306a1138981b400da68cf31e03afaaca'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:57.692729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1037/2072] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:57 2022] DELETE /identity/v3/users/306a1138981b400da68cf31e03afaaca => generated 0 bytes in 195 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:57.697162 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ab643-650b-4a30-8da7-4f5f41af103c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:57.697502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ab643-650b-4a30-8da7-4f5f41af103c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:57.697812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087ab643-650b-4a30-8da7-4f5f41af103c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:57.716523 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-087ab643-650b-4a30-8da7-4f5f41af103c None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:57.717875 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-087ab643-650b-4a30-8da7-4f5f41af103c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:57.738459 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed109382-99eb-42b5-a2cd-c91e58364029 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:57.789215 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1038/2073] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:08:57.798976 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ab643-650b-4a30-8da7-4f5f41af103c None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:57.800327 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-087ab643-650b-4a30-8da7-4f5f41af103c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:57.802941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2b9eb-7602-4dd3-adba-265738f95a69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:57.803855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2b9eb-7602-4dd3-adba-265738f95a69 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:57.804179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2b9eb-7602-4dd3-adba-265738f95a69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:57.804468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3c2b9eb-7602-4dd3-adba-265738f95a69 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:57.805174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2b9eb-7602-4dd3-adba-265738f95a69 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:57.807590 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3c2b9eb-7602-4dd3-adba-265738f95a69 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:57.816921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:57.831285 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1036/2074] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:08:57 2022] POST /identity/v3/users => generated 289 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:08:57.835254 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2b9eb-7602-4dd3-adba-265738f95a69 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:57.837050 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b3c2b9eb-7602-4dd3-adba-265738f95a69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:57.848782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3c2b9eb-7602-4dd3-adba-265738f95a69 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:57.853322 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca8a82-1805-4bfd-a8d8-e3f5212e579f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:57.854498 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca8a82-1805-4bfd-a8d8-e3f5212e579f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:57.854498 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca8a82-1805-4bfd-a8d8-e3f5212e579f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:57.854703 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ca8a82-1805-4bfd-a8d8-e3f5212e579f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:57.855263 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca8a82-1805-4bfd-a8d8-e3f5212e579f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:57.856927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ca8a82-1805-4bfd-a8d8-e3f5212e579f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:57.857654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087ab643-650b-4a30-8da7-4f5f41af103c None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:57.861225 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5427d0-44af-4039-9544-6f1df4872b54 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:57.861882 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5427d0-44af-4039-9544-6f1df4872b54 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:57.862171 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5427d0-44af-4039-9544-6f1df4872b54 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:57.862454 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5427d0-44af-4039-9544-6f1df4872b54 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:57.867610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1037/2075] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:08:57.871750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca8a82-1805-4bfd-a8d8-e3f5212e579f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:57.871910 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5427d0-44af-4039-9544-6f1df4872b54 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:57.873343 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5427d0-44af-4039-9544-6f1df4872b54 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:57.877330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:57.885273 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ca8a82-1805-4bfd-a8d8-e3f5212e579f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:57.888975 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1039/2076] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:57.891206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2360427-14a4-432e-be23-f9e8b6682e20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:57.892147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2360427-14a4-432e-be23-f9e8b6682e20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:57.892435 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2360427-14a4-432e-be23-f9e8b6682e20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:57.892716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2360427-14a4-432e-be23-f9e8b6682e20 admin admin] PATH_INFO: `/v3/projects/9261daa2bd554894a4d7b46743d20a21/users/0f5deb8a25104c71baca58e50d68a843/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:57.909330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2360427-14a4-432e-be23-f9e8b6682e20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9261daa2bd554894a4d7b46743d20a21, user_id=0f5deb8a25104c71baca58e50d68a843, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:57.912077 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2360427-14a4-432e-be23-f9e8b6682e20 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:57.926120 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1038/2077] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:57 2022] PUT /identity/v3/projects/9261daa2bd554894a4d7b46743d20a21/users/0f5deb8a25104c71baca58e50d68a843/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:57.930043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5427d0-44af-4039-9544-6f1df4872b54 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:57.940108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5427d0-44af-4039-9544-6f1df4872b54 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:58.022629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b21bb6-12f7-4c63-97a0-e65852bf5659 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:58.023334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b21bb6-12f7-4c63-97a0-e65852bf5659 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:58.023627 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b21bb6-12f7-4c63-97a0-e65852bf5659 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:58.023940 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b21bb6-12f7-4c63-97a0-e65852bf5659 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:58.024689 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b21bb6-12f7-4c63-97a0-e65852bf5659 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:58.026350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b21bb6-12f7-4c63-97a0-e65852bf5659 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:58.037177 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1040/2078] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:08:57 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:08:58.041207 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2360427-14a4-432e-be23-f9e8b6682e20 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:58.053022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2360427-14a4-432e-be23-f9e8b6682e20 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:58.058611 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d91af-24a8-4e86-8ff7-917cf743a14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:58.059076 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d91af-24a8-4e86-8ff7-917cf743a14a admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:58.059363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d91af-24a8-4e86-8ff7-917cf743a14a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:58.059646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a7d91af-24a8-4e86-8ff7-917cf743a14a admin admin] PATH_INFO: `/v3/projects/9261daa2bd554894a4d7b46743d20a21/users/0f5deb8a25104c71baca58e50d68a843/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:58.061761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d91af-24a8-4e86-8ff7-917cf743a14a admin admin] RBAC: Authorizing `identity:create_grant(project_id=9261daa2bd554894a4d7b46743d20a21, user_id=0f5deb8a25104c71baca58e50d68a843, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:58.064512 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a7d91af-24a8-4e86-8ff7-917cf743a14a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:58.075957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1039/2079] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:08:58 2022] PUT /identity/v3/projects/9261daa2bd554894a4d7b46743d20a21/users/0f5deb8a25104c71baca58e50d68a843/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:58.087105 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3653fc-9d03-4f40-86f6-8ac8b360b16b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:58.087409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3653fc-9d03-4f40-86f6-8ac8b360b16b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:58.087700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e3653fc-9d03-4f40-86f6-8ac8b360b16b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:58.113674 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4e3653fc-9d03-4f40-86f6-8ac8b360b16b None None] MFA Rules not processed for user `0f5deb8a25104c71baca58e50d68a843`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:58.114947 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4e3653fc-9d03-4f40-86f6-8ac8b360b16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:58.180835 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d91af-24a8-4e86-8ff7-917cf743a14a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:58.182773 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6a7d91af-24a8-4e86-8ff7-917cf743a14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:58.193126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1041/2080] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:58 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:58.205493 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3653fc-9d03-4f40-86f6-8ac8b360b16b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:58.208241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4e3653fc-9d03-4f40-86f6-8ac8b360b16b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:58.259517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a7d91af-24a8-4e86-8ff7-917cf743a14a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:58.264179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f1ef841-7df6-4428-b65f-7cebed93c4cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:58.264581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e3653fc-9d03-4f40-86f6-8ac8b360b16b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:58.265181 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ef841-7df6-4428-b65f-7cebed93c4cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:58.265473 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ef841-7df6-4428-b65f-7cebed93c4cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:58.265760 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f1ef841-7df6-4428-b65f-7cebed93c4cc admin admin] PATH_INFO: `/v3/projects/5a3449bb5bef4c469973ee9e65d5a134` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:58.266961 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ef841-7df6-4428-b65f-7cebed93c4cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=5a3449bb5bef4c469973ee9e65d5a134)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:58.268365 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd761b0-d02a-4513-8d51-c58f511e4f22 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:58.268955 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f1ef841-7df6-4428-b65f-7cebed93c4cc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:58.269075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd761b0-d02a-4513-8d51-c58f511e4f22 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:58.269292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd761b0-d02a-4513-8d51-c58f511e4f22 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:58.269571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd761b0-d02a-4513-8d51-c58f511e4f22 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:58.279404 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd761b0-d02a-4513-8d51-c58f511e4f22 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:58.280888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd761b0-d02a-4513-8d51-c58f511e4f22 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:58.290992 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1042/2081] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:58 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:58.324676 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5f1ef841-7df6-4428-b65f-7cebed93c4cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5a3449bb5bef4c469973ee9e65d5a134', 'request_id': 'req-5f1ef841-7df6-4428-b65f-7cebed93c4cc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:58.334479 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5f1ef841-7df6-4428-b65f-7cebed93c4cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5a3449bb5bef4c469973ee9e65d5a134', 'request_id': 'req-5f1ef841-7df6-4428-b65f-7cebed93c4cc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:58.341576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5f1ef841-7df6-4428-b65f-7cebed93c4cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5a3449bb5bef4c469973ee9e65d5a134', 'request_id': 'req-5f1ef841-7df6-4428-b65f-7cebed93c4cc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:58.353370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5f1ef841-7df6-4428-b65f-7cebed93c4cc admin admin] The token cache is being invalidate because project 5a3449bb5bef4c469973ee9e65d5a134 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:58.353678 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5f1ef841-7df6-4428-b65f-7cebed93c4cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:58.354444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5f1ef841-7df6-4428-b65f-7cebed93c4cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:58.356506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1040/2082] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:58 2022] DELETE /identity/v3/projects/5a3449bb5bef4c469973ee9e65d5a134 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:08:58.772960 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd761b0-d02a-4513-8d51-c58f511e4f22 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:58.784023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd761b0-d02a-4513-8d51-c58f511e4f22 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:58.803089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:58.841809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a604eba-c86e-46bc-a31c-3378a08bc667 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:58.842704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a604eba-c86e-46bc-a31c-3378a08bc667 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:58.842987 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a604eba-c86e-46bc-a31c-3378a08bc667 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:58.843297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a604eba-c86e-46bc-a31c-3378a08bc667 admin admin] PATH_INFO: `/v3/users/8344101ea9c04d2292a19ad36461ba9c` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:58.844512 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a604eba-c86e-46bc-a31c-3378a08bc667 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8344101ea9c04d2292a19ad36461ba9c)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:58.846444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a604eba-c86e-46bc-a31c-3378a08bc667 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:58.895464 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9a604eba-c86e-46bc-a31c-3378a08bc667 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8344101ea9c04d2292a19ad36461ba9c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:58.896695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9a604eba-c86e-46bc-a31c-3378a08bc667 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8344101ea9c04d2292a19ad36461ba9c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:58.911867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9a604eba-c86e-46bc-a31c-3378a08bc667 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8344101ea9c04d2292a19ad36461ba9c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:58.922701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9a604eba-c86e-46bc-a31c-3378a08bc667 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8344101ea9c04d2292a19ad36461ba9c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:58.923487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9a604eba-c86e-46bc-a31c-3378a08bc667 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8344101ea9c04d2292a19ad36461ba9c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:58.936018 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1043/2083] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:58 2022] DELETE /identity/v3/users/8344101ea9c04d2292a19ad36461ba9c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:58.940619 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487e2e3-ae58-4e7b-9dfc-a230d5f0184d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:58.941102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487e2e3-ae58-4e7b-9dfc-a230d5f0184d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:58.941197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6487e2e3-ae58-4e7b-9dfc-a230d5f0184d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:58.959129 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6487e2e3-ae58-4e7b-9dfc-a230d5f0184d None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:08:58.960316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6487e2e3-ae58-4e7b-9dfc-a230d5f0184d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:58.991599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:INSERT=4,keystone:SELECT=89,keystone:DELETE=7 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:59.025743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1041/2084] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:08:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:08:59.035723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a604eba-c86e-46bc-a31c-3378a08bc667 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:59.037393 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9a604eba-c86e-46bc-a31c-3378a08bc667 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:59.095560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a604eba-c86e-46bc-a31c-3378a08bc667 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:59.100640 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a92606-791c-44b3-885f-e83b47e0b03e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:59.101576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a92606-791c-44b3-885f-e83b47e0b03e service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:59.101991 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a92606-791c-44b3-885f-e83b47e0b03e service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:59.102741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2a92606-791c-44b3-885f-e83b47e0b03e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:59.114617 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a92606-791c-44b3-885f-e83b47e0b03e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:59.116128 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2a92606-791c-44b3-885f-e83b47e0b03e service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:59.126715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1044/2085] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:08:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:08:59.423378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487e2e3-ae58-4e7b-9dfc-a230d5f0184d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:59.424973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6487e2e3-ae58-4e7b-9dfc-a230d5f0184d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:08:59.441458 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6487e2e3-ae58-4e7b-9dfc-a230d5f0184d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:59.452568 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6aba93-4c22-4a37-9322-7d859926982d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:59.453633 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6aba93-4c22-4a37-9322-7d859926982d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:59.455347 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6aba93-4c22-4a37-9322-7d859926982d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:59.455347 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa6aba93-4c22-4a37-9322-7d859926982d admin admin] PATH_INFO: `/v3/projects/4a8e487eb8b04c3c9a3466aaaa8fb47d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:59.455577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6aba93-4c22-4a37-9322-7d859926982d admin admin] RBAC: Authorizing `identity:delete_project(project_id=4a8e487eb8b04c3c9a3466aaaa8fb47d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:59.457483 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa6aba93-4c22-4a37-9322-7d859926982d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:59.516466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fa6aba93-4c22-4a37-9322-7d859926982d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4a8e487eb8b04c3c9a3466aaaa8fb47d', 'request_id': 'req-fa6aba93-4c22-4a37-9322-7d859926982d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:59.525262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fa6aba93-4c22-4a37-9322-7d859926982d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4a8e487eb8b04c3c9a3466aaaa8fb47d', 'request_id': 'req-fa6aba93-4c22-4a37-9322-7d859926982d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:59.532145 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fa6aba93-4c22-4a37-9322-7d859926982d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4a8e487eb8b04c3c9a3466aaaa8fb47d', 'request_id': 'req-fa6aba93-4c22-4a37-9322-7d859926982d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:59.542813 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fa6aba93-4c22-4a37-9322-7d859926982d admin admin] The token cache is being invalidate because project 4a8e487eb8b04c3c9a3466aaaa8fb47d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:08:59.543110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fa6aba93-4c22-4a37-9322-7d859926982d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:59.543753 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fa6aba93-4c22-4a37-9322-7d859926982d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:59.545415 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1042/2086] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:08:59 2022] DELETE /identity/v3/projects/4a8e487eb8b04c3c9a3466aaaa8fb47d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:08:59.549686 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a92606-791c-44b3-885f-e83b47e0b03e service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:08:59.562194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2a92606-791c-44b3-885f-e83b47e0b03e service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:08:59.611424 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:08:59.627433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32715ff-9629-4171-a845-e46746be7806 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:08:59.628371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32715ff-9629-4171-a845-e46746be7806 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:08:59.628664 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32715ff-9629-4171-a845-e46746be7806 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:08:59.628960 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32715ff-9629-4171-a845-e46746be7806 admin admin] PATH_INFO: `/v3/users/0f5deb8a25104c71baca58e50d68a843` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:08:59.630228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32715ff-9629-4171-a845-e46746be7806 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f5deb8a25104c71baca58e50d68a843)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:08:59.632213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32715ff-9629-4171-a845-e46746be7806 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:08:59.683712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f32715ff-9629-4171-a845-e46746be7806 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f5deb8a25104c71baca58e50d68a843'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:59.684405 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f32715ff-9629-4171-a845-e46746be7806 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f5deb8a25104c71baca58e50d68a843'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:59.696305 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f32715ff-9629-4171-a845-e46746be7806 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f5deb8a25104c71baca58e50d68a843'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:59.704996 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f32715ff-9629-4171-a845-e46746be7806 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f5deb8a25104c71baca58e50d68a843'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:59.705650 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f32715ff-9629-4171-a845-e46746be7806 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f5deb8a25104c71baca58e50d68a843'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:08:59.718400 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1045/2087] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:08:59 2022] DELETE /identity/v3/users/0f5deb8a25104c71baca58e50d68a843 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:00.027807 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6aba93-4c22-4a37-9322-7d859926982d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:00.030006 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fa6aba93-4c22-4a37-9322-7d859926982d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:00.044529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa6aba93-4c22-4a37-9322-7d859926982d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:00.107809 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-922cb2da-44f3-46da-b937-748e57596f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:00.108826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cb2da-44f3-46da-b937-748e57596f13 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:00.109133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cb2da-44f3-46da-b937-748e57596f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:00.109504 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-922cb2da-44f3-46da-b937-748e57596f13 admin admin] PATH_INFO: `/v3/projects/9261daa2bd554894a4d7b46743d20a21` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:00.115723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cb2da-44f3-46da-b937-748e57596f13 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9261daa2bd554894a4d7b46743d20a21)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:00.124019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-922cb2da-44f3-46da-b937-748e57596f13 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:00.166380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-922cb2da-44f3-46da-b937-748e57596f13 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9261daa2bd554894a4d7b46743d20a21', 'request_id': 'req-922cb2da-44f3-46da-b937-748e57596f13'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:00.177402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-922cb2da-44f3-46da-b937-748e57596f13 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9261daa2bd554894a4d7b46743d20a21', 'request_id': 'req-922cb2da-44f3-46da-b937-748e57596f13'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:00.183708 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-922cb2da-44f3-46da-b937-748e57596f13 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9261daa2bd554894a4d7b46743d20a21', 'request_id': 'req-922cb2da-44f3-46da-b937-748e57596f13'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:00.185105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:00.199811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-922cb2da-44f3-46da-b937-748e57596f13 admin admin] The token cache is being invalidate because project 9261daa2bd554894a4d7b46743d20a21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:09:00.200287 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-922cb2da-44f3-46da-b937-748e57596f13 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:00.201085 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-922cb2da-44f3-46da-b937-748e57596f13 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:00.203429 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1043/2088] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:09:00 2022] DELETE /identity/v3/projects/9261daa2bd554894a4d7b46743d20a21 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:00.208654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c7108-194f-44e5-9a15-1556697d9691 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:00.209172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c7108-194f-44e5-9a15-1556697d9691 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:00.209311 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-315c7108-194f-44e5-9a15-1556697d9691 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:00.230121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-315c7108-194f-44e5-9a15-1556697d9691 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:00.231382 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-315c7108-194f-44e5-9a15-1556697d9691 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:00.320733 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1046/2089] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:00.328772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf42b0-d62f-4bc5-bc63-2efd806aed40 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:00.329030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf42b0-d62f-4bc5-bc63-2efd806aed40 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:00.329349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96bf42b0-d62f-4bc5-bc63-2efd806aed40 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:00.349584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-96bf42b0-d62f-4bc5-bc63-2efd806aed40 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:00.350834 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-96bf42b0-d62f-4bc5-bc63-2efd806aed40 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:00.359676 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1044/2090] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:00.364174 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c7108-194f-44e5-9a15-1556697d9691 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:00.374502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-315c7108-194f-44e5-9a15-1556697d9691 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:00.379309 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96e483-717e-4c03-8aaf-5b5746ecfa43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:00.380138 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96e483-717e-4c03-8aaf-5b5746ecfa43 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:00.380431 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96e483-717e-4c03-8aaf-5b5746ecfa43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:00.380709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe96e483-717e-4c03-8aaf-5b5746ecfa43 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:00.381459 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96e483-717e-4c03-8aaf-5b5746ecfa43 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:00.381772 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96e483-717e-4c03-8aaf-5b5746ecfa43 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:00.383374 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe96e483-717e-4c03-8aaf-5b5746ecfa43 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:00.397932 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1047/2091] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:09:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:09:00.403121 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf42b0-d62f-4bc5-bc63-2efd806aed40 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:00.413315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96bf42b0-d62f-4bc5-bc63-2efd806aed40 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:00.419656 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0a77e-57e9-4b9a-9b8a-569b478adb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:00.420435 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0a77e-57e9-4b9a-9b8a-569b478adb14 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:00.420722 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0a77e-57e9-4b9a-9b8a-569b478adb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:00.421002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11a0a77e-57e9-4b9a-9b8a-569b478adb14 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:00.421742 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0a77e-57e9-4b9a-9b8a-569b478adb14 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:00.423716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11a0a77e-57e9-4b9a-9b8a-569b478adb14 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:00.456485 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1045/2092] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:09:00 2022] POST /identity/v3/projects => generated 363 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:00.460884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96e483-717e-4c03-8aaf-5b5746ecfa43 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:00.471380 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe96e483-717e-4c03-8aaf-5b5746ecfa43 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:00.525335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:00.538261 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c953ac1-6efc-4e0e-8b01-f30aabbdda97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:00.538950 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c953ac1-6efc-4e0e-8b01-f30aabbdda97 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:00.539238 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c953ac1-6efc-4e0e-8b01-f30aabbdda97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:00.539526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c953ac1-6efc-4e0e-8b01-f30aabbdda97 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:00.540729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c953ac1-6efc-4e0e-8b01-f30aabbdda97 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:00.542664 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c953ac1-6efc-4e0e-8b01-f30aabbdda97 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:00.563712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1048/2093] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:09:00 2022] POST /identity/v3/users => generated 282 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:09:00.567731 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0a77e-57e9-4b9a-9b8a-569b478adb14 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:00.578404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11a0a77e-57e9-4b9a-9b8a-569b478adb14 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:00.583496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212112d-2ab7-4165-a70e-57f74c09a4e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:00.584264 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212112d-2ab7-4165-a70e-57f74c09a4e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:00.584558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212112d-2ab7-4165-a70e-57f74c09a4e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:00.584824 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f212112d-2ab7-4165-a70e-57f74c09a4e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:00.585586 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212112d-2ab7-4165-a70e-57f74c09a4e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:00.587302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f212112d-2ab7-4165-a70e-57f74c09a4e0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:00.602224 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1046/2094] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:00.606843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c953ac1-6efc-4e0e-8b01-f30aabbdda97 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:00.619701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c953ac1-6efc-4e0e-8b01-f30aabbdda97 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:00.624461 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5564830-08f6-495b-98e3-aa6975d4334a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:00.625153 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5564830-08f6-495b-98e3-aa6975d4334a admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:00.625476 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5564830-08f6-495b-98e3-aa6975d4334a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:00.625760 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5564830-08f6-495b-98e3-aa6975d4334a admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:00.626498 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5564830-08f6-495b-98e3-aa6975d4334a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:00.628169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5564830-08f6-495b-98e3-aa6975d4334a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:00.640347 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1049/2095] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:00.644039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212112d-2ab7-4165-a70e-57f74c09a4e0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:00.655385 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f212112d-2ab7-4165-a70e-57f74c09a4e0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:00.660331 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745edea8-e84d-46a2-8fa1-a67582c5f584 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:00.661229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745edea8-e84d-46a2-8fa1-a67582c5f584 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:00.661526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745edea8-e84d-46a2-8fa1-a67582c5f584 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:00.661809 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-745edea8-e84d-46a2-8fa1-a67582c5f584 admin admin] PATH_INFO: `/v3/projects/8715f5f6e59a4a74bd40bb15435d9e30/users/0bdc434dcf32435c88f9d9cc6d30105b/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:00.677173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745edea8-e84d-46a2-8fa1-a67582c5f584 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8715f5f6e59a4a74bd40bb15435d9e30, user_id=0bdc434dcf32435c88f9d9cc6d30105b, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:00.679972 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-745edea8-e84d-46a2-8fa1-a67582c5f584 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:00.695758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1047/2096] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:09:00 2022] PUT /identity/v3/projects/8715f5f6e59a4a74bd40bb15435d9e30/users/0bdc434dcf32435c88f9d9cc6d30105b/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:00.707556 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc0c4b-d4b4-4b03-97fe-27255127f27f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:00.707880 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc0c4b-d4b4-4b03-97fe-27255127f27f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:00.708168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31cc0c4b-d4b4-4b03-97fe-27255127f27f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:00.735569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-31cc0c4b-d4b4-4b03-97fe-27255127f27f None None] MFA Rules not processed for user `0bdc434dcf32435c88f9d9cc6d30105b`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:00.736817 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-31cc0c4b-d4b4-4b03-97fe-27255127f27f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:00.806319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1050/2097] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:00 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:00.810972 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745edea8-e84d-46a2-8fa1-a67582c5f584 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:00.876953 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-745edea8-e84d-46a2-8fa1-a67582c5f584 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:00.881920 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020ac89-c17b-4426-99bb-25191c0aaafb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:00.882589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020ac89-c17b-4426-99bb-25191c0aaafb admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:00.882878 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020ac89-c17b-4426-99bb-25191c0aaafb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:00.883160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f020ac89-c17b-4426-99bb-25191c0aaafb admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:00.883888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020ac89-c17b-4426-99bb-25191c0aaafb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:00.885828 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f020ac89-c17b-4426-99bb-25191c0aaafb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:00.912651 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1048/2098] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:09:00 2022] POST /identity/v3/projects => generated 365 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:00.916886 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc0c4b-d4b4-4b03-97fe-27255127f27f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:00.928428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31cc0c4b-d4b4-4b03-97fe-27255127f27f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:01.000960 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09037342-d4fc-43b9-9dba-60086d98719b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:01.002286 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09037342-d4fc-43b9-9dba-60086d98719b admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:01.002800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09037342-d4fc-43b9-9dba-60086d98719b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:01.003188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09037342-d4fc-43b9-9dba-60086d98719b admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:01.004392 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09037342-d4fc-43b9-9dba-60086d98719b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:01.007190 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09037342-d4fc-43b9-9dba-60086d98719b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:01.031068 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1051/2099] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:09:00 2022] POST /identity/v3/users => generated 289 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:01.035933 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020ac89-c17b-4426-99bb-25191c0aaafb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:01.047295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f020ac89-c17b-4426-99bb-25191c0aaafb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:01.053133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af509997-61ad-4c38-99f4-feb9ed0e9ab0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:01.053803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af509997-61ad-4c38-99f4-feb9ed0e9ab0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:01.054088 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af509997-61ad-4c38-99f4-feb9ed0e9ab0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:01.054372 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af509997-61ad-4c38-99f4-feb9ed0e9ab0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:01.055141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af509997-61ad-4c38-99f4-feb9ed0e9ab0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:01.056854 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af509997-61ad-4c38-99f4-feb9ed0e9ab0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:01.067613 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1049/2100] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:01.072205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09037342-d4fc-43b9-9dba-60086d98719b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:01.082562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09037342-d4fc-43b9-9dba-60086d98719b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:01.087560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcc162-a528-41a3-ae41-e361fd6c7671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:01.088483 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcc162-a528-41a3-ae41-e361fd6c7671 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:01.088779 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcc162-a528-41a3-ae41-e361fd6c7671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:01.089056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01dcc162-a528-41a3-ae41-e361fd6c7671 admin admin] PATH_INFO: `/v3/projects/53bc57ac27d54a06a845ac7cebaa8ebb/users/c9a0ae4442e2497cab82f057f216a694/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:01.104262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dcc162-a528-41a3-ae41-e361fd6c7671 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53bc57ac27d54a06a845ac7cebaa8ebb, user_id=c9a0ae4442e2497cab82f057f216a694, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:01.106929 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01dcc162-a528-41a3-ae41-e361fd6c7671 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:01.122379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1052/2101] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:09:01 2022] PUT /identity/v3/projects/53bc57ac27d54a06a845ac7cebaa8ebb/users/c9a0ae4442e2497cab82f057f216a694/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:01.127926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af509997-61ad-4c38-99f4-feb9ed0e9ab0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:01.141942 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af509997-61ad-4c38-99f4-feb9ed0e9ab0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:01.204505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0933ed1-2f23-43b9-9820-17b98a812e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:01.205190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0933ed1-2f23-43b9-9820-17b98a812e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:01.205479 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0933ed1-2f23-43b9-9820-17b98a812e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:01.205761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0933ed1-2f23-43b9-9820-17b98a812e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:01.206502 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0933ed1-2f23-43b9-9820-17b98a812e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:01.208173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0933ed1-2f23-43b9-9820-17b98a812e8a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:01.209686 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=2,keystone:SELECT=95,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:01.220130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1050/2102] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:01 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:01.224592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcc162-a528-41a3-ae41-e361fd6c7671 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:01.235551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01dcc162-a528-41a3-ae41-e361fd6c7671 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:01.240385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531f913-6011-481a-91b1-aebed6bf5c43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:01.241447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531f913-6011-481a-91b1-aebed6bf5c43 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:01.241735 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531f913-6011-481a-91b1-aebed6bf5c43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:01.242027 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0531f913-6011-481a-91b1-aebed6bf5c43 admin admin] PATH_INFO: `/v3/projects/53bc57ac27d54a06a845ac7cebaa8ebb/users/c9a0ae4442e2497cab82f057f216a694/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:01.244367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0531f913-6011-481a-91b1-aebed6bf5c43 admin admin] RBAC: Authorizing `identity:create_grant(project_id=53bc57ac27d54a06a845ac7cebaa8ebb, user_id=c9a0ae4442e2497cab82f057f216a694, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:01.247074 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0531f913-6011-481a-91b1-aebed6bf5c43 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:01.267334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1053/2103] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:09:01 2022] PUT /identity/v3/projects/53bc57ac27d54a06a845ac7cebaa8ebb/users/c9a0ae4442e2497cab82f057f216a694/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:01.277766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a1147-7fb3-4cf0-bfc4-6d5d93bec23c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:01.278064 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a1147-7fb3-4cf0-bfc4-6d5d93bec23c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:01.278425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682a1147-7fb3-4cf0-bfc4-6d5d93bec23c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:01.312076 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-682a1147-7fb3-4cf0-bfc4-6d5d93bec23c None None] MFA Rules not processed for user `c9a0ae4442e2497cab82f057f216a694`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:01.313322 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-682a1147-7fb3-4cf0-bfc4-6d5d93bec23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:01.383781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1051/2104] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:01 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:01.396175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531f913-6011-481a-91b1-aebed6bf5c43 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:01.398251 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0531f913-6011-481a-91b1-aebed6bf5c43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:01.413599 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:01.465999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0531f913-6011-481a-91b1-aebed6bf5c43 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:01.470008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488aabc9-df3a-42ae-b5c0-8334edbce2b8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:01.470668 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488aabc9-df3a-42ae-b5c0-8334edbce2b8 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:01.470959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488aabc9-df3a-42ae-b5c0-8334edbce2b8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:01.471242 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488aabc9-df3a-42ae-b5c0-8334edbce2b8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:01.482141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488aabc9-df3a-42ae-b5c0-8334edbce2b8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:01.483646 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-488aabc9-df3a-42ae-b5c0-8334edbce2b8 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:01.496246 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1054/2105] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:01 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:09:01.582721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a1147-7fb3-4cf0-bfc4-6d5d93bec23c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:01.584378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-682a1147-7fb3-4cf0-bfc4-6d5d93bec23c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:01.597513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682a1147-7fb3-4cf0-bfc4-6d5d93bec23c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:01.601306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3888b88-f367-4b05-ad59-81b6682988f6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:01.601973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3888b88-f367-4b05-ad59-81b6682988f6 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:01.602261 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3888b88-f367-4b05-ad59-81b6682988f6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:01.602547 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3888b88-f367-4b05-ad59-81b6682988f6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:01.657080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3888b88-f367-4b05-ad59-81b6682988f6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:01.658416 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3888b88-f367-4b05-ad59-81b6682988f6 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:01.667497 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1052/2106] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:01 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:09:11.489088 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=21 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:11.663341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=64 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:13.097925 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488aabc9-df3a-42ae-b5c0-8334edbce2b8 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:13.164109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488aabc9-df3a-42ae-b5c0-8334edbce2b8 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:13.170219 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5019ea-cdb6-408f-a8ba-1d22fca77e1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:13.171156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5019ea-cdb6-408f-a8ba-1d22fca77e1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:13.171448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5019ea-cdb6-408f-a8ba-1d22fca77e1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:13.171729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f5019ea-cdb6-408f-a8ba-1d22fca77e1b admin admin] PATH_INFO: `/v3/users/0bdc434dcf32435c88f9d9cc6d30105b` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:13.173141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5019ea-cdb6-408f-a8ba-1d22fca77e1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=0bdc434dcf32435c88f9d9cc6d30105b)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:13.175153 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f5019ea-cdb6-408f-a8ba-1d22fca77e1b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:13.230597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0f5019ea-cdb6-408f-a8ba-1d22fca77e1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0bdc434dcf32435c88f9d9cc6d30105b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:13.231261 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0f5019ea-cdb6-408f-a8ba-1d22fca77e1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0bdc434dcf32435c88f9d9cc6d30105b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:13.243157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0f5019ea-cdb6-408f-a8ba-1d22fca77e1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0bdc434dcf32435c88f9d9cc6d30105b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:13.251657 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0f5019ea-cdb6-408f-a8ba-1d22fca77e1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0bdc434dcf32435c88f9d9cc6d30105b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:13.252296 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0f5019ea-cdb6-408f-a8ba-1d22fca77e1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0bdc434dcf32435c88f9d9cc6d30105b'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:13.264395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1055/2107] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:09:13 2022] DELETE /identity/v3/users/0bdc434dcf32435c88f9d9cc6d30105b => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:13.269039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1fc8a-536f-4917-8213-91a7245e2fe9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:13.269527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1fc8a-536f-4917-8213-91a7245e2fe9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:13.269622 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c1fc8a-536f-4917-8213-91a7245e2fe9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:13.289383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-86c1fc8a-536f-4917-8213-91a7245e2fe9 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:13.290669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-86c1fc8a-536f-4917-8213-91a7245e2fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:13.357914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1053/2108] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:13.367802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5019ea-cdb6-408f-a8ba-1d22fca77e1b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:13.369386 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0f5019ea-cdb6-408f-a8ba-1d22fca77e1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:13.430799 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f5019ea-cdb6-408f-a8ba-1d22fca77e1b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:13.435124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e5b1e-092b-4ec3-8836-14faa57870ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:13.435778 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e5b1e-092b-4ec3-8836-14faa57870ce service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:13.436097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e5b1e-092b-4ec3-8836-14faa57870ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:13.436375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063e5b1e-092b-4ec3-8836-14faa57870ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:13.446050 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e5b1e-092b-4ec3-8836-14faa57870ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:13.447505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-063e5b1e-092b-4ec3-8836-14faa57870ce service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:13.457513 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1056/2109] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:09:13.749081 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1fc8a-536f-4917-8213-91a7245e2fe9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:13.750630 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-86c1fc8a-536f-4917-8213-91a7245e2fe9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:13.764644 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c1fc8a-536f-4917-8213-91a7245e2fe9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:13.772079 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38ea07-fb01-494e-af21-99bd5246db9d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:13.773009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38ea07-fb01-494e-af21-99bd5246db9d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:13.773304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38ea07-fb01-494e-af21-99bd5246db9d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:13.773590 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af38ea07-fb01-494e-af21-99bd5246db9d admin admin] PATH_INFO: `/v3/projects/8715f5f6e59a4a74bd40bb15435d9e30` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:13.774934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af38ea07-fb01-494e-af21-99bd5246db9d admin admin] RBAC: Authorizing `identity:delete_project(project_id=8715f5f6e59a4a74bd40bb15435d9e30)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:13.777258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af38ea07-fb01-494e-af21-99bd5246db9d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:13.835927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af38ea07-fb01-494e-af21-99bd5246db9d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8715f5f6e59a4a74bd40bb15435d9e30', 'request_id': 'req-af38ea07-fb01-494e-af21-99bd5246db9d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:13.844480 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af38ea07-fb01-494e-af21-99bd5246db9d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8715f5f6e59a4a74bd40bb15435d9e30', 'request_id': 'req-af38ea07-fb01-494e-af21-99bd5246db9d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:13.851138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af38ea07-fb01-494e-af21-99bd5246db9d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8715f5f6e59a4a74bd40bb15435d9e30', 'request_id': 'req-af38ea07-fb01-494e-af21-99bd5246db9d'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:13.862562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af38ea07-fb01-494e-af21-99bd5246db9d admin admin] The token cache is being invalidate because project 8715f5f6e59a4a74bd40bb15435d9e30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:09:13.862865 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af38ea07-fb01-494e-af21-99bd5246db9d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:13.863527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-af38ea07-fb01-494e-af21-99bd5246db9d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:13.865107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1054/2110] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:09:13 2022] DELETE /identity/v3/projects/8715f5f6e59a4a74bd40bb15435d9e30 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:13.869191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e5b1e-092b-4ec3-8836-14faa57870ce service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:13.879127 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063e5b1e-092b-4ec3-8836-14faa57870ce service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:13.898163 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:13.941963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09eee1be-d6b0-4b27-ab69-831312d146ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:13.942875 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eee1be-d6b0-4b27-ab69-831312d146ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:13.943161 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eee1be-d6b0-4b27-ab69-831312d146ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:13.943444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09eee1be-d6b0-4b27-ab69-831312d146ac admin admin] PATH_INFO: `/v3/users/c9a0ae4442e2497cab82f057f216a694` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:13.944708 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eee1be-d6b0-4b27-ab69-831312d146ac admin admin] RBAC: Authorizing `identity:delete_user(user_id=c9a0ae4442e2497cab82f057f216a694)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:13.946687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09eee1be-d6b0-4b27-ab69-831312d146ac admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:14.012288 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-09eee1be-d6b0-4b27-ab69-831312d146ac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c9a0ae4442e2497cab82f057f216a694'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:14.012933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-09eee1be-d6b0-4b27-ab69-831312d146ac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c9a0ae4442e2497cab82f057f216a694'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:14.025181 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-09eee1be-d6b0-4b27-ab69-831312d146ac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c9a0ae4442e2497cab82f057f216a694'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:14.034992 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-09eee1be-d6b0-4b27-ab69-831312d146ac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c9a0ae4442e2497cab82f057f216a694'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:14.035649 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-09eee1be-d6b0-4b27-ab69-831312d146ac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c9a0ae4442e2497cab82f057f216a694'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:14.048785 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1057/2111] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:09:13 2022] DELETE /identity/v3/users/c9a0ae4442e2497cab82f057f216a694 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:14.400860 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38ea07-fb01-494e-af21-99bd5246db9d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:14.402694 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-af38ea07-fb01-494e-af21-99bd5246db9d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:14.416160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af38ea07-fb01-494e-af21-99bd5246db9d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:14.477139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6887c7be-ffaf-4a48-b9f0-e6d6182045cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:14.478106 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887c7be-ffaf-4a48-b9f0-e6d6182045cc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:14.478392 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887c7be-ffaf-4a48-b9f0-e6d6182045cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:14.478695 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6887c7be-ffaf-4a48-b9f0-e6d6182045cc admin admin] PATH_INFO: `/v3/projects/53bc57ac27d54a06a845ac7cebaa8ebb` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:14.479967 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887c7be-ffaf-4a48-b9f0-e6d6182045cc admin admin] RBAC: Authorizing `identity:delete_project(project_id=53bc57ac27d54a06a845ac7cebaa8ebb)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:14.482016 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6887c7be-ffaf-4a48-b9f0-e6d6182045cc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:14.517841 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:14.541288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6887c7be-ffaf-4a48-b9f0-e6d6182045cc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '53bc57ac27d54a06a845ac7cebaa8ebb', 'request_id': 'req-6887c7be-ffaf-4a48-b9f0-e6d6182045cc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:14.550786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6887c7be-ffaf-4a48-b9f0-e6d6182045cc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '53bc57ac27d54a06a845ac7cebaa8ebb', 'request_id': 'req-6887c7be-ffaf-4a48-b9f0-e6d6182045cc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:14.557623 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6887c7be-ffaf-4a48-b9f0-e6d6182045cc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '53bc57ac27d54a06a845ac7cebaa8ebb', 'request_id': 'req-6887c7be-ffaf-4a48-b9f0-e6d6182045cc'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:14.568785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6887c7be-ffaf-4a48-b9f0-e6d6182045cc admin admin] The token cache is being invalidate because project 53bc57ac27d54a06a845ac7cebaa8ebb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:09:14.569087 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6887c7be-ffaf-4a48-b9f0-e6d6182045cc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:14.569802 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6887c7be-ffaf-4a48-b9f0-e6d6182045cc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:14.571691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1055/2112] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:09:14 2022] DELETE /identity/v3/projects/53bc57ac27d54a06a845ac7cebaa8ebb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:14.577037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23879531-6c03-411f-9a01-372b892e7ad4 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:14.577497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23879531-6c03-411f-9a01-372b892e7ad4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:14.577621 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23879531-6c03-411f-9a01-372b892e7ad4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:14.598455 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-23879531-6c03-411f-9a01-372b892e7ad4 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:14.599672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-23879531-6c03-411f-9a01-372b892e7ad4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:14.672940 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1058/2113] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:14.682280 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6460b3-43ab-4330-ae7e-8f1864b921e6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:14.682590 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6460b3-43ab-4330-ae7e-8f1864b921e6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:14.683048 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6460b3-43ab-4330-ae7e-8f1864b921e6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:14.703107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9b6460b3-43ab-4330-ae7e-8f1864b921e6 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:14.704388 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9b6460b3-43ab-4330-ae7e-8f1864b921e6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:14.713656 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1056/2114] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:14.718234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23879531-6c03-411f-9a01-372b892e7ad4 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:14.729145 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23879531-6c03-411f-9a01-372b892e7ad4 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:14.735089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0533e17-c761-4b60-99c8-5dc1cfa603b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:14.735774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0533e17-c761-4b60-99c8-5dc1cfa603b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:14.736097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0533e17-c761-4b60-99c8-5dc1cfa603b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:14.736380 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0533e17-c761-4b60-99c8-5dc1cfa603b6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:14.737156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0533e17-c761-4b60-99c8-5dc1cfa603b6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:14.737468 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0533e17-c761-4b60-99c8-5dc1cfa603b6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:14.739142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0533e17-c761-4b60-99c8-5dc1cfa603b6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:14.751446 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1059/2115] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:09:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:09:14.755782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6460b3-43ab-4330-ae7e-8f1864b921e6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:14.765357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6460b3-43ab-4330-ae7e-8f1864b921e6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:14.770334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d5e9f-0355-4e9d-8676-786f73982d5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:14.770960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d5e9f-0355-4e9d-8676-786f73982d5d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:14.771253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d5e9f-0355-4e9d-8676-786f73982d5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:14.771538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d5e9f-0355-4e9d-8676-786f73982d5d admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:14.772305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d5e9f-0355-4e9d-8676-786f73982d5d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:14.774250 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d5e9f-0355-4e9d-8676-786f73982d5d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:14.803210 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1057/2116] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:09:14 2022] POST /identity/v3/projects => generated 383 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:14.807662 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0533e17-c761-4b60-99c8-5dc1cfa603b6 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:14.818302 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0533e17-c761-4b60-99c8-5dc1cfa603b6 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:14.846126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:14.881310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcf99b-f943-463b-94e6-84d57805315b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:14.882000 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcf99b-f943-463b-94e6-84d57805315b admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:14.882292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcf99b-f943-463b-94e6-84d57805315b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:14.882579 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15dcf99b-f943-463b-94e6-84d57805315b admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:14.883278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dcf99b-f943-463b-94e6-84d57805315b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:14.885284 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15dcf99b-f943-463b-94e6-84d57805315b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:14.908210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1060/2117] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:09:14 2022] POST /identity/v3/users => generated 292 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:14.912565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d5e9f-0355-4e9d-8676-786f73982d5d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:14.923241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d5e9f-0355-4e9d-8676-786f73982d5d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:14.928359 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c5a96-dada-4423-934c-b7ee9be84e0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:14.929024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c5a96-dada-4423-934c-b7ee9be84e0c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:14.929304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c5a96-dada-4423-934c-b7ee9be84e0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:14.929593 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c5a96-dada-4423-934c-b7ee9be84e0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:14.930334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c5a96-dada-4423-934c-b7ee9be84e0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:14.932059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c5a96-dada-4423-934c-b7ee9be84e0c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:14.943000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1058/2118] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:14.948743 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcf99b-f943-463b-94e6-84d57805315b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:14.960164 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15dcf99b-f943-463b-94e6-84d57805315b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:14.965325 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-065ca7d2-7277-4093-99a1-6fb5da5b88a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:14.965987 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ca7d2-7277-4093-99a1-6fb5da5b88a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:14.966281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ca7d2-7277-4093-99a1-6fb5da5b88a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:14.966561 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-065ca7d2-7277-4093-99a1-6fb5da5b88a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:14.967297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ca7d2-7277-4093-99a1-6fb5da5b88a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:14.968967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-065ca7d2-7277-4093-99a1-6fb5da5b88a9 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:14.979461 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1061/2119] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:14.984150 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c5a96-dada-4423-934c-b7ee9be84e0c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:14.995335 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c5a96-dada-4423-934c-b7ee9be84e0c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:15.000878 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5f0c8-9385-4e9d-9865-9142b402f949 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:15.001884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5f0c8-9385-4e9d-9865-9142b402f949 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:15.002168 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5f0c8-9385-4e9d-9865-9142b402f949 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:15.002447 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b5f0c8-9385-4e9d-9865-9142b402f949 admin admin] PATH_INFO: `/v3/projects/96b574e2a4624394a341450358b18bf7/users/37e0a6e0f14944b49c53b794c54ff2de/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:15.019665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5f0c8-9385-4e9d-9865-9142b402f949 admin admin] RBAC: Authorizing `identity:create_grant(project_id=96b574e2a4624394a341450358b18bf7, user_id=37e0a6e0f14944b49c53b794c54ff2de, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:15.022423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43b5f0c8-9385-4e9d-9865-9142b402f949 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:15.038558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1059/2120] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:09:14 2022] PUT /identity/v3/projects/96b574e2a4624394a341450358b18bf7/users/37e0a6e0f14944b49c53b794c54ff2de/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:15.049963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e9a0e-d60d-4a90-8d94-1760230e9ac8 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:15.050264 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e9a0e-d60d-4a90-8d94-1760230e9ac8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:15.050554 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696e9a0e-d60d-4a90-8d94-1760230e9ac8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:15.081947 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-696e9a0e-d60d-4a90-8d94-1760230e9ac8 None None] MFA Rules not processed for user `37e0a6e0f14944b49c53b794c54ff2de`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:15.084033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-696e9a0e-d60d-4a90-8d94-1760230e9ac8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:15.154392 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1062/2121] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:15 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:15.159436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5f0c8-9385-4e9d-9865-9142b402f949 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:15.224374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b5f0c8-9385-4e9d-9865-9142b402f949 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:15.230035 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1ffae-dff5-42fa-ae2c-24f90fe9d532 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:15.230704 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1ffae-dff5-42fa-ae2c-24f90fe9d532 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:15.230993 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1ffae-dff5-42fa-ae2c-24f90fe9d532 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:15.231297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee1ffae-dff5-42fa-ae2c-24f90fe9d532 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:15.232033 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1ffae-dff5-42fa-ae2c-24f90fe9d532 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:15.233968 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee1ffae-dff5-42fa-ae2c-24f90fe9d532 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:15.264475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1060/2122] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:09:15 2022] POST /identity/v3/projects => generated 379 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:15.269421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e9a0e-d60d-4a90-8d94-1760230e9ac8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:15.281019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696e9a0e-d60d-4a90-8d94-1760230e9ac8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:15.350411 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedcce6-a385-4228-a9c5-24b1f1de39fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:15.351131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedcce6-a385-4228-a9c5-24b1f1de39fa admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:15.351375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedcce6-a385-4228-a9c5-24b1f1de39fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:15.352084 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcedcce6-a385-4228-a9c5-24b1f1de39fa admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:15.352861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcedcce6-a385-4228-a9c5-24b1f1de39fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:15.354848 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcedcce6-a385-4228-a9c5-24b1f1de39fa admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:15.378561 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1063/2123] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:09:15 2022] POST /identity/v3/users => generated 296 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:15.382560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1ffae-dff5-42fa-ae2c-24f90fe9d532 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:15.393780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee1ffae-dff5-42fa-ae2c-24f90fe9d532 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:15.397860 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f3706-4843-47dd-882b-8dc289559b95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:15.398556 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f3706-4843-47dd-882b-8dc289559b95 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:15.398845 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f3706-4843-47dd-882b-8dc289559b95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:15.399131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f52f3706-4843-47dd-882b-8dc289559b95 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:15.400134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f3706-4843-47dd-882b-8dc289559b95 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:15.402173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f52f3706-4843-47dd-882b-8dc289559b95 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:15.413850 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1061/2124] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:15.417898 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedcce6-a385-4228-a9c5-24b1f1de39fa admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:15.428667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcedcce6-a385-4228-a9c5-24b1f1de39fa admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:15.434531 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69613434-00fc-4e17-ba44-d294ebfcaeff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:15.435472 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69613434-00fc-4e17-ba44-d294ebfcaeff admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:15.435762 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69613434-00fc-4e17-ba44-d294ebfcaeff admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:15.436075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69613434-00fc-4e17-ba44-d294ebfcaeff admin admin] PATH_INFO: `/v3/projects/9a1360b64a434d58a3a5d7420a84a513/users/a045d06f45b049d58315aa440ae56cd0/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:15.460960 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69613434-00fc-4e17-ba44-d294ebfcaeff admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1360b64a434d58a3a5d7420a84a513, user_id=a045d06f45b049d58315aa440ae56cd0, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:15.464290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69613434-00fc-4e17-ba44-d294ebfcaeff admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:15.482117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1064/2125] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:09:15 2022] PUT /identity/v3/projects/9a1360b64a434d58a3a5d7420a84a513/users/a045d06f45b049d58315aa440ae56cd0/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:15.485931 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f3706-4843-47dd-882b-8dc289559b95 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:15.496016 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f52f3706-4843-47dd-882b-8dc289559b95 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:15.516600 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:15.558861 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb3108f-870a-40eb-81b3-1cc2a69ddf27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:15.559548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3108f-870a-40eb-81b3-1cc2a69ddf27 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:15.559852 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3108f-870a-40eb-81b3-1cc2a69ddf27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:15.560159 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb3108f-870a-40eb-81b3-1cc2a69ddf27 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:15.560899 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3108f-870a-40eb-81b3-1cc2a69ddf27 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:15.562509 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb3108f-870a-40eb-81b3-1cc2a69ddf27 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:15.572941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1062/2126] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:15 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:15.579460 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69613434-00fc-4e17-ba44-d294ebfcaeff admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:15.587477 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:15.593807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69613434-00fc-4e17-ba44-d294ebfcaeff admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:15.598897 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30258f1-d06e-4f62-8887-5816afd59499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:15.599792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30258f1-d06e-4f62-8887-5816afd59499 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:15.600109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30258f1-d06e-4f62-8887-5816afd59499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:15.600395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f30258f1-d06e-4f62-8887-5816afd59499 admin admin] PATH_INFO: `/v3/projects/9a1360b64a434d58a3a5d7420a84a513/users/a045d06f45b049d58315aa440ae56cd0/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:15.602390 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30258f1-d06e-4f62-8887-5816afd59499 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a1360b64a434d58a3a5d7420a84a513, user_id=a045d06f45b049d58315aa440ae56cd0, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:15.606032 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f30258f1-d06e-4f62-8887-5816afd59499 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:15.616417 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1065/2127] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:09:15 2022] PUT /identity/v3/projects/9a1360b64a434d58a3a5d7420a84a513/users/a045d06f45b049d58315aa440ae56cd0/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:15.626573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c76148-e91e-4b43-a69f-6377ce6cd216 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:15.626876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c76148-e91e-4b43-a69f-6377ce6cd216 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:15.627168 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c76148-e91e-4b43-a69f-6377ce6cd216 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:15.653996 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-30c76148-e91e-4b43-a69f-6377ce6cd216 None None] MFA Rules not processed for user `a045d06f45b049d58315aa440ae56cd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:15.655212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-30c76148-e91e-4b43-a69f-6377ce6cd216 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:15.733667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1063/2128] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:15 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:15.744717 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30258f1-d06e-4f62-8887-5816afd59499 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:15.746760 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f30258f1-d06e-4f62-8887-5816afd59499 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:15.807286 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f30258f1-d06e-4f62-8887-5816afd59499 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:15.811069 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33bbe6a-1b8b-4831-8c51-272c74193759 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:15.811710 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bbe6a-1b8b-4831-8c51-272c74193759 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:15.812031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bbe6a-1b8b-4831-8c51-272c74193759 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:15.812314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b33bbe6a-1b8b-4831-8c51-272c74193759 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:15.821467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33bbe6a-1b8b-4831-8c51-272c74193759 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:15.822903 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b33bbe6a-1b8b-4831-8c51-272c74193759 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:15.832512 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1066/2129] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:15 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:09:15.937984 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c76148-e91e-4b43-a69f-6377ce6cd216 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:15.947772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c76148-e91e-4b43-a69f-6377ce6cd216 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:15.951573 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-292d007f-8562-4109-805c-8c5afcc50f16 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:15.952257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d007f-8562-4109-805c-8c5afcc50f16 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:15.952547 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d007f-8562-4109-805c-8c5afcc50f16 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:15.952827 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-292d007f-8562-4109-805c-8c5afcc50f16 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:16.009877 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d007f-8562-4109-805c-8c5afcc50f16 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:16.011323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-292d007f-8562-4109-805c-8c5afcc50f16 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:16.020615 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1064/2130] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:15 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:09:18.600369 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33bbe6a-1b8b-4831-8c51-272c74193759 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:18.664715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b33bbe6a-1b8b-4831-8c51-272c74193759 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:18.669549 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cda6a5-763a-4baa-86ca-1cb7b33338e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:18.670439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cda6a5-763a-4baa-86ca-1cb7b33338e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:18.670760 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cda6a5-763a-4baa-86ca-1cb7b33338e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:18.671039 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53cda6a5-763a-4baa-86ca-1cb7b33338e4 admin admin] PATH_INFO: `/v3/users/37e0a6e0f14944b49c53b794c54ff2de` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:18.672277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cda6a5-763a-4baa-86ca-1cb7b33338e4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=37e0a6e0f14944b49c53b794c54ff2de)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:18.674234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53cda6a5-763a-4baa-86ca-1cb7b33338e4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:18.731010 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-53cda6a5-763a-4baa-86ca-1cb7b33338e4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '37e0a6e0f14944b49c53b794c54ff2de'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:18.731703 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-53cda6a5-763a-4baa-86ca-1cb7b33338e4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '37e0a6e0f14944b49c53b794c54ff2de'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:18.745202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-53cda6a5-763a-4baa-86ca-1cb7b33338e4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '37e0a6e0f14944b49c53b794c54ff2de'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:18.755395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-53cda6a5-763a-4baa-86ca-1cb7b33338e4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '37e0a6e0f14944b49c53b794c54ff2de'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:18.756049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-53cda6a5-763a-4baa-86ca-1cb7b33338e4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '37e0a6e0f14944b49c53b794c54ff2de'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:18.769820 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1067/2131] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:09:18 2022] DELETE /identity/v3/users/37e0a6e0f14944b49c53b794c54ff2de => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:18.774642 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca60df-62b5-4c98-b6f8-a758431ada04 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:18.775099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca60df-62b5-4c98-b6f8-a758431ada04 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:18.775226 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca60df-62b5-4c98-b6f8-a758431ada04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:18.793043 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9eca60df-62b5-4c98-b6f8-a758431ada04 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:18.794149 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9eca60df-62b5-4c98-b6f8-a758431ada04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:18.844285 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:18.863737 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1065/2132] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:18.873963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cda6a5-763a-4baa-86ca-1cb7b33338e4 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:18.875615 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-53cda6a5-763a-4baa-86ca-1cb7b33338e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:18.889337 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:18.937167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53cda6a5-763a-4baa-86ca-1cb7b33338e4 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:18.940956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a24438-2882-4a1a-aaa6-e500b08e9951 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:18.941617 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a24438-2882-4a1a-aaa6-e500b08e9951 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:18.941905 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a24438-2882-4a1a-aaa6-e500b08e9951 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:18.942181 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78a24438-2882-4a1a-aaa6-e500b08e9951 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:18.951028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a24438-2882-4a1a-aaa6-e500b08e9951 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:18.952505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78a24438-2882-4a1a-aaa6-e500b08e9951 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:18.961991 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1068/2133] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:09:19.238809 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca60df-62b5-4c98-b6f8-a758431ada04 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:19.240361 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9eca60df-62b5-4c98-b6f8-a758431ada04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:19.252651 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca60df-62b5-4c98-b6f8-a758431ada04 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:19.257429 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:19.258315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:19.258603 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:19.258924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9 admin admin] PATH_INFO: `/v3/projects/96b574e2a4624394a341450358b18bf7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:19.260161 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=96b574e2a4624394a341450358b18bf7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:19.262065 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:19.312600 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96b574e2a4624394a341450358b18bf7', 'request_id': 'req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:19.321499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96b574e2a4624394a341450358b18bf7', 'request_id': 'req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:19.328378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96b574e2a4624394a341450358b18bf7', 'request_id': 'req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:19.339895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9 admin admin] The token cache is being invalidate because project 96b574e2a4624394a341450358b18bf7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:09:19.340229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:19.341282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:19.342503 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1066/2134] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:09:19 2022] DELETE /identity/v3/projects/96b574e2a4624394a341450358b18bf7 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:19.346818 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a24438-2882-4a1a-aaa6-e500b08e9951 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:19.356576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78a24438-2882-4a1a-aaa6-e500b08e9951 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:19.414077 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38df86f8-6eed-4b78-90d6-83ee77b016de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:19.415018 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df86f8-6eed-4b78-90d6-83ee77b016de admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:19.415330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df86f8-6eed-4b78-90d6-83ee77b016de admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:19.415667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38df86f8-6eed-4b78-90d6-83ee77b016de admin admin] PATH_INFO: `/v3/users/a045d06f45b049d58315aa440ae56cd0` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:19.417044 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df86f8-6eed-4b78-90d6-83ee77b016de admin admin] RBAC: Authorizing `identity:delete_user(user_id=a045d06f45b049d58315aa440ae56cd0)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:19.418996 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38df86f8-6eed-4b78-90d6-83ee77b016de admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:19.466788 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-38df86f8-6eed-4b78-90d6-83ee77b016de admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a045d06f45b049d58315aa440ae56cd0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:19.467536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-38df86f8-6eed-4b78-90d6-83ee77b016de admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a045d06f45b049d58315aa440ae56cd0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:19.480348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-38df86f8-6eed-4b78-90d6-83ee77b016de admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a045d06f45b049d58315aa440ae56cd0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:19.490338 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-38df86f8-6eed-4b78-90d6-83ee77b016de admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a045d06f45b049d58315aa440ae56cd0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:19.490979 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-38df86f8-6eed-4b78-90d6-83ee77b016de admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a045d06f45b049d58315aa440ae56cd0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:19.503738 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1069/2135] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:09:19 2022] DELETE /identity/v3/users/a045d06f45b049d58315aa440ae56cd0 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:19.865072 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:19.866740 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:19.880299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2049d1ce-b502-4130-b0fc-a5d1df4bc2c9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:19.937393 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deedc28e-e4c7-4661-8845-17951ff286db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:19.938284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedc28e-e4c7-4661-8845-17951ff286db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:19.938575 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedc28e-e4c7-4661-8845-17951ff286db admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:19.938860 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deedc28e-e4c7-4661-8845-17951ff286db admin admin] PATH_INFO: `/v3/projects/9a1360b64a434d58a3a5d7420a84a513` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:19.940176 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedc28e-e4c7-4661-8845-17951ff286db admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a1360b64a434d58a3a5d7420a84a513)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:19.942114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deedc28e-e4c7-4661-8845-17951ff286db admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:19.992259 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-deedc28e-e4c7-4661-8845-17951ff286db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a1360b64a434d58a3a5d7420a84a513', 'request_id': 'req-deedc28e-e4c7-4661-8845-17951ff286db'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:20.002239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-deedc28e-e4c7-4661-8845-17951ff286db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a1360b64a434d58a3a5d7420a84a513', 'request_id': 'req-deedc28e-e4c7-4661-8845-17951ff286db'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:20.009154 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-deedc28e-e4c7-4661-8845-17951ff286db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a1360b64a434d58a3a5d7420a84a513', 'request_id': 'req-deedc28e-e4c7-4661-8845-17951ff286db'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:20.015918 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:20.021817 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-deedc28e-e4c7-4661-8845-17951ff286db admin admin] The token cache is being invalidate because project 9a1360b64a434d58a3a5d7420a84a513 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:09:20.022190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-deedc28e-e4c7-4661-8845-17951ff286db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:20.023215 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-deedc28e-e4c7-4661-8845-17951ff286db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:20.025606 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1067/2136] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:09:19 2022] DELETE /identity/v3/projects/9a1360b64a434d58a3a5d7420a84a513 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:09:20.030961 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76161ccc-df6f-4ef8-8bcb-9d8570c67769 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:20.031471 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76161ccc-df6f-4ef8-8bcb-9d8570c67769 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:20.031577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76161ccc-df6f-4ef8-8bcb-9d8570c67769 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:20.049493 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-76161ccc-df6f-4ef8-8bcb-9d8570c67769 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:20.050709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-76161ccc-df6f-4ef8-8bcb-9d8570c67769 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:20.080386 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:20.112894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1070/2137] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:20.120966 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8761f-0579-4ed8-9801-93239828b699 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:20.121269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8761f-0579-4ed8-9801-93239828b699 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:20.121563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb8761f-0579-4ed8-9801-93239828b699 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:20.141296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9cb8761f-0579-4ed8-9801-93239828b699 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:20.142505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9cb8761f-0579-4ed8-9801-93239828b699 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:20.151236 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1068/2138] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:20.155792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76161ccc-df6f-4ef8-8bcb-9d8570c67769 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:20.166368 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76161ccc-df6f-4ef8-8bcb-9d8570c67769 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:20.172049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab6d32-30f3-4310-9ba9-d02db55fab86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:20.172727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab6d32-30f3-4310-9ba9-d02db55fab86 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:20.173020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab6d32-30f3-4310-9ba9-d02db55fab86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:20.173302 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cab6d32-30f3-4310-9ba9-d02db55fab86 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:20.174044 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab6d32-30f3-4310-9ba9-d02db55fab86 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:20.174361 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab6d32-30f3-4310-9ba9-d02db55fab86 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:20.176046 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cab6d32-30f3-4310-9ba9-d02db55fab86 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:20.188909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1071/2139] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:09:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:09:20.192611 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8761f-0579-4ed8-9801-93239828b699 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:20.202184 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb8761f-0579-4ed8-9801-93239828b699 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:20.207177 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa203f3c-31cc-4221-9bea-6dacb8b6eb0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:20.207852 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa203f3c-31cc-4221-9bea-6dacb8b6eb0f admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:20.208160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa203f3c-31cc-4221-9bea-6dacb8b6eb0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:20.208444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa203f3c-31cc-4221-9bea-6dacb8b6eb0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:20.209152 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa203f3c-31cc-4221-9bea-6dacb8b6eb0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:20.211083 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa203f3c-31cc-4221-9bea-6dacb8b6eb0f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:20.239639 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1069/2140] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:09:20 2022] POST /identity/v3/projects => generated 367 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:20.244082 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab6d32-30f3-4310-9ba9-d02db55fab86 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:20.254445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cab6d32-30f3-4310-9ba9-d02db55fab86 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:20.314564 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85484ef4-303b-4562-8998-463bcaa15387 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:20.315301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85484ef4-303b-4562-8998-463bcaa15387 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:20.315587 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85484ef4-303b-4562-8998-463bcaa15387 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:20.315880 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85484ef4-303b-4562-8998-463bcaa15387 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:20.316577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85484ef4-303b-4562-8998-463bcaa15387 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:20.318502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85484ef4-303b-4562-8998-463bcaa15387 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:20.340813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1072/2141] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:09:20 2022] POST /identity/v3/users => generated 284 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:20.344972 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa203f3c-31cc-4221-9bea-6dacb8b6eb0f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:20.356190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa203f3c-31cc-4221-9bea-6dacb8b6eb0f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:20.361547 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efc3e5-0917-4269-9879-078742678a5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:20.362211 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efc3e5-0917-4269-9879-078742678a5f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:20.362504 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efc3e5-0917-4269-9879-078742678a5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:20.362790 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95efc3e5-0917-4269-9879-078742678a5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:20.363534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efc3e5-0917-4269-9879-078742678a5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:20.365204 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95efc3e5-0917-4269-9879-078742678a5f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:20.375931 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1070/2142] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:09:20.379530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85484ef4-303b-4562-8998-463bcaa15387 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:20.389673 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85484ef4-303b-4562-8998-463bcaa15387 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:20.395909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d63f2e8-72de-45a8-9c6d-15af598b43eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:20.396573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63f2e8-72de-45a8-9c6d-15af598b43eb admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:20.396857 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63f2e8-72de-45a8-9c6d-15af598b43eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:20.397125 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d63f2e8-72de-45a8-9c6d-15af598b43eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:20.397853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63f2e8-72de-45a8-9c6d-15af598b43eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:20.399436 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d63f2e8-72de-45a8-9c6d-15af598b43eb admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:20.410810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1073/2143] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:20 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:20.415138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efc3e5-0917-4269-9879-078742678a5f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:20.425570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95efc3e5-0917-4269-9879-078742678a5f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:20.430970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61b148-73aa-45a0-b301-3592ab53d27f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:20.431843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61b148-73aa-45a0-b301-3592ab53d27f admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:20.432169 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61b148-73aa-45a0-b301-3592ab53d27f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:20.432446 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d61b148-73aa-45a0-b301-3592ab53d27f admin admin] PATH_INFO: `/v3/projects/f7f047df998e495798bf2660814db661/users/b99619a058ce4e9d9c014a45bac91e07/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:20.447742 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61b148-73aa-45a0-b301-3592ab53d27f admin admin] RBAC: Authorizing `identity:create_grant(project_id=f7f047df998e495798bf2660814db661, user_id=b99619a058ce4e9d9c014a45bac91e07, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:20.450533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d61b148-73aa-45a0-b301-3592ab53d27f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:20.465734 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1071/2144] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:09:20 2022] PUT /identity/v3/projects/f7f047df998e495798bf2660814db661/users/b99619a058ce4e9d9c014a45bac91e07/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:20.471799 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163a341-db48-499a-a904-0916d0c3cf49 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:20.472262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163a341-db48-499a-a904-0916d0c3cf49 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:20.472624 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163a341-db48-499a-a904-0916d0c3cf49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:20.492716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8163a341-db48-499a-a904-0916d0c3cf49 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:20.493952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8163a341-db48-499a-a904-0916d0c3cf49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:20.555854 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1074/2145] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:09:20.565500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61b148-73aa-45a0-b301-3592ab53d27f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:20.567606 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4d61b148-73aa-45a0-b301-3592ab53d27f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:20.633170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d61b148-73aa-45a0-b301-3592ab53d27f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:20.640458 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c6603-cbd3-4cef-ad39-87521ea66b27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:20.641470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c6603-cbd3-4cef-ad39-87521ea66b27 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:20.641909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c6603-cbd3-4cef-ad39-87521ea66b27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:20.642355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6c6603-cbd3-4cef-ad39-87521ea66b27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:20.655585 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c6603-cbd3-4cef-ad39-87521ea66b27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:20.657089 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6c6603-cbd3-4cef-ad39-87521ea66b27 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:20.666541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1072/2146] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:09:21.381278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45827dd3-ab42-4ec6-be19-b5334c2261a2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:21.381811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45827dd3-ab42-4ec6-be19-b5334c2261a2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:21.381913 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45827dd3-ab42-4ec6-be19-b5334c2261a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:21.409172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-45827dd3-ab42-4ec6-be19-b5334c2261a2 None None] MFA Rules not processed for user `b99619a058ce4e9d9c014a45bac91e07`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:21.410372 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-45827dd3-ab42-4ec6-be19-b5334c2261a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:21.440073 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:21.475860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1075/2147] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:21 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:21.480693 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c6603-cbd3-4cef-ad39-87521ea66b27 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:21.490501 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6c6603-cbd3-4cef-ad39-87521ea66b27 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:21.495299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ff18f-17bb-4871-8b91-9d16f464817d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:21.496044 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ff18f-17bb-4871-8b91-9d16f464817d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:21.496334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ff18f-17bb-4871-8b91-9d16f464817d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:21.496615 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ff18f-17bb-4871-8b91-9d16f464817d admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:21.497319 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ff18f-17bb-4871-8b91-9d16f464817d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:21.499234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ff18f-17bb-4871-8b91-9d16f464817d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:21.528338 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1073/2148] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:09:21 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:21.529810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45827dd3-ab42-4ec6-be19-b5334c2261a2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:21.540762 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45827dd3-ab42-4ec6-be19-b5334c2261a2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:21.595092 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee70c7d-26ef-4c62-8347-3d50ca8a4068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:21.595769 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee70c7d-26ef-4c62-8347-3d50ca8a4068 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:21.596088 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee70c7d-26ef-4c62-8347-3d50ca8a4068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:21.596362 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee70c7d-26ef-4c62-8347-3d50ca8a4068 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:21.597049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee70c7d-26ef-4c62-8347-3d50ca8a4068 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:21.598982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bee70c7d-26ef-4c62-8347-3d50ca8a4068 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:21.624850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1076/2149] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:09:21 2022] POST /identity/v3/users => generated 292 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:21.626307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ff18f-17bb-4871-8b91-9d16f464817d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:21.636451 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ff18f-17bb-4871-8b91-9d16f464817d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:21.641229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473d87e-1889-41d0-8480-326660092dc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:21.641934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473d87e-1889-41d0-8480-326660092dc0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:21.642233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473d87e-1889-41d0-8480-326660092dc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:21.642547 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a473d87e-1889-41d0-8480-326660092dc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:21.643294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473d87e-1889-41d0-8480-326660092dc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:21.644950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a473d87e-1889-41d0-8480-326660092dc0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:21.656027 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1074/2150] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:21.660108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee70c7d-26ef-4c62-8347-3d50ca8a4068 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:21.670273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee70c7d-26ef-4c62-8347-3d50ca8a4068 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:21.675777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b78796-5119-4a4e-b50a-23b8cd4ab7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:21.676704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b78796-5119-4a4e-b50a-23b8cd4ab7d2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:21.676987 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b78796-5119-4a4e-b50a-23b8cd4ab7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:21.677266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4b78796-5119-4a4e-b50a-23b8cd4ab7d2 admin admin] PATH_INFO: `/v3/projects/d97854d072824236bfa880a44fa3aced/users/10e6196f4293496ba39524b3aa881169/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:21.692986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b78796-5119-4a4e-b50a-23b8cd4ab7d2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d97854d072824236bfa880a44fa3aced, user_id=10e6196f4293496ba39524b3aa881169, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:21.695665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4b78796-5119-4a4e-b50a-23b8cd4ab7d2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:21.710759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1077/2151] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:09:21 2022] PUT /identity/v3/projects/d97854d072824236bfa880a44fa3aced/users/10e6196f4293496ba39524b3aa881169/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:09:21.714888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473d87e-1889-41d0-8480-326660092dc0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:21.724995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a473d87e-1889-41d0-8480-326660092dc0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:21.791565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7adac03b-3332-427e-a380-33dcc41d4e5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:21.792251 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adac03b-3332-427e-a380-33dcc41d4e5f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:21.792569 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adac03b-3332-427e-a380-33dcc41d4e5f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:21.792854 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7adac03b-3332-427e-a380-33dcc41d4e5f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:21.793589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adac03b-3332-427e-a380-33dcc41d4e5f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:21.795219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7adac03b-3332-427e-a380-33dcc41d4e5f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:21.800629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:21.806171 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1075/2152] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:21 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:21.810616 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b78796-5119-4a4e-b50a-23b8cd4ab7d2 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:21.821390 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4b78796-5119-4a4e-b50a-23b8cd4ab7d2 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:21.826356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d38bd-b668-46e2-8d2f-ac8a0a33b5cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:21.827244 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d38bd-b668-46e2-8d2f-ac8a0a33b5cd admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:21.827531 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d38bd-b668-46e2-8d2f-ac8a0a33b5cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:21.827811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32d38bd-b668-46e2-8d2f-ac8a0a33b5cd admin admin] PATH_INFO: `/v3/projects/d97854d072824236bfa880a44fa3aced/users/10e6196f4293496ba39524b3aa881169/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:21.829853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d38bd-b668-46e2-8d2f-ac8a0a33b5cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=d97854d072824236bfa880a44fa3aced, user_id=10e6196f4293496ba39524b3aa881169, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:21.832541 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d32d38bd-b668-46e2-8d2f-ac8a0a33b5cd admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:21.842727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1078/2153] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:09:21 2022] PUT /identity/v3/projects/d97854d072824236bfa880a44fa3aced/users/10e6196f4293496ba39524b3aa881169/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:22.471170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425ec1-1b2f-4522-8696-a4830444323d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:22.471473 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425ec1-1b2f-4522-8696-a4830444323d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:22.471763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e425ec1-1b2f-4522-8696-a4830444323d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:22.499716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4e425ec1-1b2f-4522-8696-a4830444323d None None] MFA Rules not processed for user `10e6196f4293496ba39524b3aa881169`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:22.500971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4e425ec1-1b2f-4522-8696-a4830444323d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:22.578421 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1076/2154] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:22 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:22.590292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d38bd-b668-46e2-8d2f-ac8a0a33b5cd admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:22.593002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d32d38bd-b668-46e2-8d2f-ac8a0a33b5cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:22.662615 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d32d38bd-b668-46e2-8d2f-ac8a0a33b5cd admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:22.669467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f09e28-a05c-43ba-a994-0e393dff3781 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:22.670157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f09e28-a05c-43ba-a994-0e393dff3781 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:22.670450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f09e28-a05c-43ba-a994-0e393dff3781 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:22.670739 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f09e28-a05c-43ba-a994-0e393dff3781 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:22.715168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:22.744538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f09e28-a05c-43ba-a994-0e393dff3781 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:22.746849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f09e28-a05c-43ba-a994-0e393dff3781 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:22.761135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1079/2155] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:22 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:09:26.243738 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e425ec1-1b2f-4522-8696-a4830444323d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:26.245330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4e425ec1-1b2f-4522-8696-a4830444323d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:26.258360 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e425ec1-1b2f-4522-8696-a4830444323d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:26.262811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904658c8-aa93-468a-bb8e-02d61ca671e7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:26.263461 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904658c8-aa93-468a-bb8e-02d61ca671e7 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:26.263746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904658c8-aa93-468a-bb8e-02d61ca671e7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:26.264046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904658c8-aa93-468a-bb8e-02d61ca671e7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:26.279467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904658c8-aa93-468a-bb8e-02d61ca671e7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:26.281066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904658c8-aa93-468a-bb8e-02d61ca671e7 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:26.291297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1077/2156] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:26 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:09:32.755274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=11 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:36.287353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=40 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:39.876902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f09e28-a05c-43ba-a994-0e393dff3781 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:39.878390 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f7f09e28-a05c-43ba-a994-0e393dff3781 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:39.945505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f09e28-a05c-43ba-a994-0e393dff3781 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:39.950304 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c81c5-00eb-4b90-a6cc-4137cc31e81c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:39.951203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c81c5-00eb-4b90-a6cc-4137cc31e81c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:39.951496 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c81c5-00eb-4b90-a6cc-4137cc31e81c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:39.951781 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4c81c5-00eb-4b90-a6cc-4137cc31e81c admin admin] PATH_INFO: `/v3/users/b2459b9a6a3c495a82702b184656f664` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:39.953278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c81c5-00eb-4b90-a6cc-4137cc31e81c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b2459b9a6a3c495a82702b184656f664)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:39.955266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d4c81c5-00eb-4b90-a6cc-4137cc31e81c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:40.002605 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6d4c81c5-00eb-4b90-a6cc-4137cc31e81c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b2459b9a6a3c495a82702b184656f664'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:40.003346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6d4c81c5-00eb-4b90-a6cc-4137cc31e81c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b2459b9a6a3c495a82702b184656f664'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:40.015046 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6d4c81c5-00eb-4b90-a6cc-4137cc31e81c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b2459b9a6a3c495a82702b184656f664'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:40.024046 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6d4c81c5-00eb-4b90-a6cc-4137cc31e81c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b2459b9a6a3c495a82702b184656f664'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:40.025062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6d4c81c5-00eb-4b90-a6cc-4137cc31e81c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b2459b9a6a3c495a82702b184656f664'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:40.036908 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1080/2157] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:09:39 2022] DELETE /identity/v3/users/b2459b9a6a3c495a82702b184656f664 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:40.041584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f7e8c-53ce-481a-97a1-36768f951fa6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:40.042094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f7e8c-53ce-481a-97a1-36768f951fa6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:40.042094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8f7e8c-53ce-481a-97a1-36768f951fa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:40.083991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-db8f7e8c-53ce-481a-97a1-36768f951fa6 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:40.085330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-db8f7e8c-53ce-481a-97a1-36768f951fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:40.097709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c81c5-00eb-4b90-a6cc-4137cc31e81c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:40.099205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6d4c81c5-00eb-4b90-a6cc-4137cc31e81c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:40.147079 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1078/2158] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:40.157172 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f7e8c-53ce-481a-97a1-36768f951fa6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:40.158674 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-db8f7e8c-53ce-481a-97a1-36768f951fa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:40.184043 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4c81c5-00eb-4b90-a6cc-4137cc31e81c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:40.190572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:40.219086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8f7e8c-53ce-481a-97a1-36768f951fa6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:40.220682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1fded-6131-4a4b-9cd1-c472e06fefe4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:40.221792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1fded-6131-4a4b-9cd1-c472e06fefe4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:40.222069 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1fded-6131-4a4b-9cd1-c472e06fefe4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:40.222417 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5e1fded-6131-4a4b-9cd1-c472e06fefe4 admin admin] PATH_INFO: `/v3/users/b7db815d31cf4c54845f5f8c9dcb1b17` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:40.222955 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1f164b-16c5-400c-af72-704aba9e569a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:40.223711 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1fded-6131-4a4b-9cd1-c472e06fefe4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7db815d31cf4c54845f5f8c9dcb1b17)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:40.223858 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f164b-16c5-400c-af72-704aba9e569a service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:40.224083 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f164b-16c5-400c-af72-704aba9e569a service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:40.224368 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1f164b-16c5-400c-af72-704aba9e569a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:40.225704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5e1fded-6131-4a4b-9cd1-c472e06fefe4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:40.234138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f164b-16c5-400c-af72-704aba9e569a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:40.235592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1f164b-16c5-400c-af72-704aba9e569a service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:40.245670 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1079/2159] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:09:40.276341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e5e1fded-6131-4a4b-9cd1-c472e06fefe4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7db815d31cf4c54845f5f8c9dcb1b17'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:40.277144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e5e1fded-6131-4a4b-9cd1-c472e06fefe4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7db815d31cf4c54845f5f8c9dcb1b17'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:40.290374 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e5e1fded-6131-4a4b-9cd1-c472e06fefe4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7db815d31cf4c54845f5f8c9dcb1b17'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:40.301067 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e5e1fded-6131-4a4b-9cd1-c472e06fefe4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7db815d31cf4c54845f5f8c9dcb1b17'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:40.302015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e5e1fded-6131-4a4b-9cd1-c472e06fefe4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7db815d31cf4c54845f5f8c9dcb1b17'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:40.318736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1081/2160] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:09:40 2022] DELETE /identity/v3/users/b7db815d31cf4c54845f5f8c9dcb1b17 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:40.323333 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91cfe5-bd49-4647-8e42-73f9e5b0293a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:40.323664 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91cfe5-bd49-4647-8e42-73f9e5b0293a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:40.323995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f91cfe5-bd49-4647-8e42-73f9e5b0293a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:40.351679 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6f91cfe5-bd49-4647-8e42-73f9e5b0293a None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:40.352970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6f91cfe5-bd49-4647-8e42-73f9e5b0293a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:40.416931 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1080/2161] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:40.426778 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1fded-6131-4a4b-9cd1-c472e06fefe4 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:40.428316 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e5e1fded-6131-4a4b-9cd1-c472e06fefe4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:40.515398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91cfe5-bd49-4647-8e42-73f9e5b0293a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:40.517338 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6f91cfe5-bd49-4647-8e42-73f9e5b0293a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:40.530722 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5e1fded-6131-4a4b-9cd1-c472e06fefe4 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:40.532795 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f91cfe5-bd49-4647-8e42-73f9e5b0293a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:40.534808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2565710a-5e8e-4114-ac08-58ebe6cca8ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:40.535410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565710a-5e8e-4114-ac08-58ebe6cca8ca service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:40.535640 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565710a-5e8e-4114-ac08-58ebe6cca8ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:40.535955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2565710a-5e8e-4114-ac08-58ebe6cca8ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:40.538176 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a04c63cb-1e3a-44cb-a992-4e1e64b87db8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:40.539057 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c63cb-1e3a-44cb-a992-4e1e64b87db8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:40.539350 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c63cb-1e3a-44cb-a992-4e1e64b87db8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:40.539867 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04c63cb-1e3a-44cb-a992-4e1e64b87db8 admin admin] PATH_INFO: `/v3/projects/e37f7240a9bd4f36a63f9c6043aecc88` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:40.541242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c63cb-1e3a-44cb-a992-4e1e64b87db8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e37f7240a9bd4f36a63f9c6043aecc88)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:40.543288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a04c63cb-1e3a-44cb-a992-4e1e64b87db8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:40.547562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565710a-5e8e-4114-ac08-58ebe6cca8ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:40.549106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2565710a-5e8e-4114-ac08-58ebe6cca8ca service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:40.552687 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:40.559663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1082/2162] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:09:40.598294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a04c63cb-1e3a-44cb-a992-4e1e64b87db8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e37f7240a9bd4f36a63f9c6043aecc88', 'request_id': 'req-a04c63cb-1e3a-44cb-a992-4e1e64b87db8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:40.607378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a04c63cb-1e3a-44cb-a992-4e1e64b87db8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e37f7240a9bd4f36a63f9c6043aecc88', 'request_id': 'req-a04c63cb-1e3a-44cb-a992-4e1e64b87db8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:40.614353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a04c63cb-1e3a-44cb-a992-4e1e64b87db8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e37f7240a9bd4f36a63f9c6043aecc88', 'request_id': 'req-a04c63cb-1e3a-44cb-a992-4e1e64b87db8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:40.626841 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a04c63cb-1e3a-44cb-a992-4e1e64b87db8 admin admin] The token cache is being invalidate because project e37f7240a9bd4f36a63f9c6043aecc88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:09:40.627141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a04c63cb-1e3a-44cb-a992-4e1e64b87db8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:40.627903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-a04c63cb-1e3a-44cb-a992-4e1e64b87db8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:40.629724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1081/2163] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:09:40 2022] DELETE /identity/v3/projects/e37f7240a9bd4f36a63f9c6043aecc88 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:09:40.634907 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6c2a-73c9-492e-9e3f-802305bbbfd0 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:40.635300 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6c2a-73c9-492e-9e3f-802305bbbfd0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:40.635682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5e6c2a-73c9-492e-9e3f-802305bbbfd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:40.678896 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-1e5e6c2a-73c9-492e-9e3f-802305bbbfd0 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:40.679944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1e5e6c2a-73c9-492e-9e3f-802305bbbfd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:40.685504 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:40.753475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1083/2164] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:40.755447 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42612af3-77bf-45ff-8f0f-a2d5e4a4747b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:40.755969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42612af3-77bf-45ff-8f0f-a2d5e4a4747b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:40.756410 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42612af3-77bf-45ff-8f0f-a2d5e4a4747b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:40.777124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-42612af3-77bf-45ff-8f0f-a2d5e4a4747b None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:40.778378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-42612af3-77bf-45ff-8f0f-a2d5e4a4747b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:40.790441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1082/2165] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:40.791983 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e6c2a-73c9-492e-9e3f-802305bbbfd0 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:40.803137 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5e6c2a-73c9-492e-9e3f-802305bbbfd0 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:40.812698 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f4ebf-16a7-450e-baeb-bd85c08ca623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:40.813866 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f4ebf-16a7-450e-baeb-bd85c08ca623 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:40.814403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f4ebf-16a7-450e-baeb-bd85c08ca623 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:40.814530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a19f4ebf-16a7-450e-baeb-bd85c08ca623 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:40.815172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19f4ebf-16a7-450e-baeb-bd85c08ca623 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:40.815487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19f4ebf-16a7-450e-baeb-bd85c08ca623 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:40.818091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a19f4ebf-16a7-450e-baeb-bd85c08ca623 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:40.834247 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1084/2166] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:09:40.840029 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42612af3-77bf-45ff-8f0f-a2d5e4a4747b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:40.855097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42612af3-77bf-45ff-8f0f-a2d5e4a4747b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:40.862198 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baeab41-5860-40ea-ae9d-d89b3b340559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:40.862984 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeab41-5860-40ea-ae9d-d89b3b340559 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:40.863270 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeab41-5860-40ea-ae9d-d89b3b340559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:40.863582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1baeab41-5860-40ea-ae9d-d89b3b340559 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:40.864319 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baeab41-5860-40ea-ae9d-d89b3b340559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:40.866252 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1baeab41-5860-40ea-ae9d-d89b3b340559 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:40.879064 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f4ebf-16a7-450e-baeb-bd85c08ca623 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:40.884660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a19f4ebf-16a7-450e-baeb-bd85c08ca623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:40.899957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a19f4ebf-16a7-450e-baeb-bd85c08ca623 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:40.922414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276ca5d6-bf83-4054-8dbb-3e75f808c446 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:40.923614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ca5d6-bf83-4054-8dbb-3e75f808c446 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:40.924012 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ca5d6-bf83-4054-8dbb-3e75f808c446 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:40.927421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276ca5d6-bf83-4054-8dbb-3e75f808c446 admin admin] PATH_INFO: `/v3/projects/57f9464c88474b21a98f614f3d7a911c` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:40.928012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1083/2167] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:09:40 2022] POST /identity/v3/projects => generated 381 bytes in 89 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:40.938034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baeab41-5860-40ea-ae9d-d89b3b340559 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:40.940663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ca5d6-bf83-4054-8dbb-3e75f808c446 admin admin] RBAC: Authorizing `identity:delete_project(project_id=57f9464c88474b21a98f614f3d7a911c)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:40.945401 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276ca5d6-bf83-4054-8dbb-3e75f808c446 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:41.035773 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1baeab41-5860-40ea-ae9d-d89b3b340559 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:41.053372 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-276ca5d6-bf83-4054-8dbb-3e75f808c446 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '57f9464c88474b21a98f614f3d7a911c', 'request_id': 'req-276ca5d6-bf83-4054-8dbb-3e75f808c446'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:41.062334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-276ca5d6-bf83-4054-8dbb-3e75f808c446 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '57f9464c88474b21a98f614f3d7a911c', 'request_id': 'req-276ca5d6-bf83-4054-8dbb-3e75f808c446'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:41.072723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-276ca5d6-bf83-4054-8dbb-3e75f808c446 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '57f9464c88474b21a98f614f3d7a911c', 'request_id': 'req-276ca5d6-bf83-4054-8dbb-3e75f808c446'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:41.086625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-276ca5d6-bf83-4054-8dbb-3e75f808c446 admin admin] The token cache is being invalidate because project 57f9464c88474b21a98f614f3d7a911c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:09:41.086921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-276ca5d6-bf83-4054-8dbb-3e75f808c446 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:41.087596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-276ca5d6-bf83-4054-8dbb-3e75f808c446 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:41.089193 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1085/2168] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:09:40 2022] DELETE /identity/v3/projects/57f9464c88474b21a98f614f3d7a911c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:41.094088 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d6f25-24c6-427a-b55b-c6eb6c156a5c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:41.094380 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d6f25-24c6-427a-b55b-c6eb6c156a5c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:41.094667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84d6f25-24c6-427a-b55b-c6eb6c156a5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:41.116020 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26bf179-14d5-456b-ad16-aa5b522f1f94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:41.116613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f84d6f25-24c6-427a-b55b-c6eb6c156a5c None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:41.116841 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bf179-14d5-456b-ad16-aa5b522f1f94 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:41.117627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bf179-14d5-456b-ad16-aa5b522f1f94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:41.117627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26bf179-14d5-456b-ad16-aa5b522f1f94 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:41.118047 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f84d6f25-24c6-427a-b55b-c6eb6c156a5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:41.118229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26bf179-14d5-456b-ad16-aa5b522f1f94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:41.120222 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a26bf179-14d5-456b-ad16-aa5b522f1f94 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:41.127623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1086/2169] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:09:41.134808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2490eac-d1d3-41d9-ae88-c386550cfb34 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:41.135293 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2490eac-d1d3-41d9-ae88-c386550cfb34 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:41.135722 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2490eac-d1d3-41d9-ae88-c386550cfb34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:41.151394 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1084/2170] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:09:40 2022] POST /identity/v3/users => generated 291 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:41.155886 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26bf179-14d5-456b-ad16-aa5b522f1f94 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:41.156796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f2490eac-d1d3-41d9-ae88-c386550cfb34 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:41.157743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a26bf179-14d5-456b-ad16-aa5b522f1f94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:41.158446 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f2490eac-d1d3-41d9-ae88-c386550cfb34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:41.167341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:41.167700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1087/2171] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:41.171498 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2490eac-d1d3-41d9-ae88-c386550cfb34 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:41.176696 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26bf179-14d5-456b-ad16-aa5b522f1f94 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:41.181444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2490eac-d1d3-41d9-ae88-c386550cfb34 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:41.185047 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd1483-3631-44a8-97e5-9efb6114baba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:41.185922 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd1483-3631-44a8-97e5-9efb6114baba admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:41.186308 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b2514-5874-4087-9789-3cc885a5e62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:41.186594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd1483-3631-44a8-97e5-9efb6114baba admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:41.187090 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b2514-5874-4087-9789-3cc885a5e62d admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:41.187216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63bd1483-3631-44a8-97e5-9efb6114baba admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:41.187390 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b2514-5874-4087-9789-3cc885a5e62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:41.187654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b2514-5874-4087-9789-3cc885a5e62d admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:41.188000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd1483-3631-44a8-97e5-9efb6114baba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:41.188542 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b2514-5874-4087-9789-3cc885a5e62d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:41.188858 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b2514-5874-4087-9789-3cc885a5e62d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:41.190195 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63bd1483-3631-44a8-97e5-9efb6114baba admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:41.190473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b2514-5874-4087-9789-3cc885a5e62d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:41.203866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1085/2172] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:41.204513 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1088/2173] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:09:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:09:41.207334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd1483-3631-44a8-97e5-9efb6114baba admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:41.208199 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b2514-5874-4087-9789-3cc885a5e62d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:41.217538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63bd1483-3631-44a8-97e5-9efb6114baba admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:41.218052 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b2514-5874-4087-9789-3cc885a5e62d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:41.224414 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1267fce-b185-40c6-b8c8-3e271933c3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:41.225082 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1267fce-b185-40c6-b8c8-3e271933c3bb admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:41.225398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1267fce-b185-40c6-b8c8-3e271933c3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:41.225506 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b67a8-e839-44a6-a7ff-40df58fa954d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:41.225684 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1267fce-b185-40c6-b8c8-3e271933c3bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:41.226098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b67a8-e839-44a6-a7ff-40df58fa954d admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:41.226391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b67a8-e839-44a6-a7ff-40df58fa954d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:41.226487 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1267fce-b185-40c6-b8c8-3e271933c3bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:41.226680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155b67a8-e839-44a6-a7ff-40df58fa954d admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:41.227444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b67a8-e839-44a6-a7ff-40df58fa954d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:41.228120 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1267fce-b185-40c6-b8c8-3e271933c3bb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:41.229384 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155b67a8-e839-44a6-a7ff-40df58fa954d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:41.238930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1086/2174] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:09:41.243195 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1267fce-b185-40c6-b8c8-3e271933c3bb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:41.253717 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1267fce-b185-40c6-b8c8-3e271933c3bb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:41.258321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d933f5-c282-4b45-851d-a7a2b177fb14 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:41.258454 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1089/2175] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:09:41 2022] POST /identity/v3/projects => generated 399 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:09:41.259212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d933f5-c282-4b45-851d-a7a2b177fb14 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:41.259529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d933f5-c282-4b45-851d-a7a2b177fb14 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:41.259813 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d933f5-c282-4b45-851d-a7a2b177fb14 admin admin] PATH_INFO: `/v3/projects/b32433ec45854be380fcd9a9671a4ce5/users/2b909d4c7d9b45b29f995268f8fac987/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:41.262667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b67a8-e839-44a6-a7ff-40df58fa954d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:41.278440 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d933f5-c282-4b45-851d-a7a2b177fb14 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b32433ec45854be380fcd9a9671a4ce5, user_id=2b909d4c7d9b45b29f995268f8fac987, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:41.281150 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d933f5-c282-4b45-851d-a7a2b177fb14 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:41.297851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1087/2176] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:09:41 2022] PUT /identity/v3/projects/b32433ec45854be380fcd9a9671a4ce5/users/2b909d4c7d9b45b29f995268f8fac987/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:09:41.298068 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:41.304044 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400e614-5e48-4c82-89b0-67525c425b25 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:41.304480 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400e614-5e48-4c82-89b0-67525c425b25 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:41.304911 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f400e614-5e48-4c82-89b0-67525c425b25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:41.324508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f400e614-5e48-4c82-89b0-67525c425b25 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:41.325920 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f400e614-5e48-4c82-89b0-67525c425b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:41.335042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155b67a8-e839-44a6-a7ff-40df58fa954d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:41.385762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1088/2177] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:41.392573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67d623-fbc1-466c-98f0-a62fa0639186 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:41.393263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67d623-fbc1-466c-98f0-a62fa0639186 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:41.393556 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67d623-fbc1-466c-98f0-a62fa0639186 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:41.393841 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c67d623-fbc1-466c-98f0-a62fa0639186 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:41.394553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67d623-fbc1-466c-98f0-a62fa0639186 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:41.396102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400e614-5e48-4c82-89b0-67525c425b25 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:41.396843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c67d623-fbc1-466c-98f0-a62fa0639186 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:41.397478 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f400e614-5e48-4c82-89b0-67525c425b25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:41.417693 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1090/2178] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:09:41 2022] POST /identity/v3/users => generated 300 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:41.421625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67d623-fbc1-466c-98f0-a62fa0639186 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:41.431580 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c67d623-fbc1-466c-98f0-a62fa0639186 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:41.436561 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d5722-50c5-4fdf-b950-c3d108d18e6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:41.437337 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d5722-50c5-4fdf-b950-c3d108d18e6a admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:41.437721 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d5722-50c5-4fdf-b950-c3d108d18e6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:41.438119 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d5722-50c5-4fdf-b950-c3d108d18e6a admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:41.439081 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d5722-50c5-4fdf-b950-c3d108d18e6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:41.441094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d5722-50c5-4fdf-b950-c3d108d18e6a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:41.451941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1091/2179] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:09:41.456065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d5722-50c5-4fdf-b950-c3d108d18e6a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:41.460716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f400e614-5e48-4c82-89b0-67525c425b25 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:41.464641 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6966a1e-1090-4273-864c-e933af0d04fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:41.465290 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6966a1e-1090-4273-864c-e933af0d04fc service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:41.465542 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d5722-50c5-4fdf-b950-c3d108d18e6a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:41.465640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6966a1e-1090-4273-864c-e933af0d04fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:41.465836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6966a1e-1090-4273-864c-e933af0d04fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:41.470047 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca4682-14fa-42d5-b508-036ae5a44b5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:41.470687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca4682-14fa-42d5-b508-036ae5a44b5e admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:41.470976 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca4682-14fa-42d5-b508-036ae5a44b5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:41.471259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ca4682-14fa-42d5-b508-036ae5a44b5e admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:41.472015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca4682-14fa-42d5-b508-036ae5a44b5e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:41.473606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30ca4682-14fa-42d5-b508-036ae5a44b5e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:41.474815 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6966a1e-1090-4273-864c-e933af0d04fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:41.476280 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6966a1e-1090-4273-864c-e933af0d04fc service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:41.484692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1092/2180] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:41.486326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1089/2181] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:09:41.489205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca4682-14fa-42d5-b508-036ae5a44b5e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:41.499155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ca4682-14fa-42d5-b508-036ae5a44b5e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:41.503723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58545d74-2b3f-4b2e-a489-b4a5b4ff6d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:41.504648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58545d74-2b3f-4b2e-a489-b4a5b4ff6d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:41.504940 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58545d74-2b3f-4b2e-a489-b4a5b4ff6d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:41.505246 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58545d74-2b3f-4b2e-a489-b4a5b4ff6d2e admin admin] PATH_INFO: `/v3/projects/7cec4cc428754579bcf1a56f555bc363/users/0decd125ebb448f5a5c92ae3212b2f70/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:41.520256 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58545d74-2b3f-4b2e-a489-b4a5b4ff6d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7cec4cc428754579bcf1a56f555bc363, user_id=0decd125ebb448f5a5c92ae3212b2f70, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:41.522943 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58545d74-2b3f-4b2e-a489-b4a5b4ff6d2e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:41.536273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1093/2182] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:09:41 2022] PUT /identity/v3/projects/7cec4cc428754579bcf1a56f555bc363/users/0decd125ebb448f5a5c92ae3212b2f70/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:09:41.548563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2b7dd-8611-4f67-a5d8-c4de278d2251 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:41.548882 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2b7dd-8611-4f67-a5d8-c4de278d2251 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:41.549296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6d2b7dd-8611-4f67-a5d8-c4de278d2251 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:41.578321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a6d2b7dd-8611-4f67-a5d8-c4de278d2251 None None] MFA Rules not processed for user `0decd125ebb448f5a5c92ae3212b2f70`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:41.579650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a6d2b7dd-8611-4f67-a5d8-c4de278d2251 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:41.631744 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:41.654934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1090/2183] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:41 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:41.668293 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58545d74-2b3f-4b2e-a489-b4a5b4ff6d2e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:41.670248 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-58545d74-2b3f-4b2e-a489-b4a5b4ff6d2e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:41.730808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58545d74-2b3f-4b2e-a489-b4a5b4ff6d2e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:41.735605 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e03ca-c544-417c-aa66-192f100fb8d4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:41.736703 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e03ca-c544-417c-aa66-192f100fb8d4 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:41.737122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e03ca-c544-417c-aa66-192f100fb8d4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:41.737410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e03ca-c544-417c-aa66-192f100fb8d4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:41.747454 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e03ca-c544-417c-aa66-192f100fb8d4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:41.750756 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e03ca-c544-417c-aa66-192f100fb8d4 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:41.759735 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1094/2184] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:41 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:09:42.210472 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2faff-86d8-4cc2-9001-0fc54a61eb75 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:42.211133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2faff-86d8-4cc2-9001-0fc54a61eb75 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:42.211133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60d2faff-86d8-4cc2-9001-0fc54a61eb75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:42.265414 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-60d2faff-86d8-4cc2-9001-0fc54a61eb75 None None] MFA Rules not processed for user `2b909d4c7d9b45b29f995268f8fac987`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:42.267433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-60d2faff-86d8-4cc2-9001-0fc54a61eb75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:42.401239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1091/2185] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:42 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:42.403633 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e03ca-c544-417c-aa66-192f100fb8d4 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:42.405851 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-079e03ca-c544-417c-aa66-192f100fb8d4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:42.423920 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e03ca-c544-417c-aa66-192f100fb8d4 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:42.429232 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af943bbf-a6ba-4c29-b88b-8d38c1b08840 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:42.430087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af943bbf-a6ba-4c29-b88b-8d38c1b08840 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:42.430478 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af943bbf-a6ba-4c29-b88b-8d38c1b08840 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:42.430764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af943bbf-a6ba-4c29-b88b-8d38c1b08840 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:42.443141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af943bbf-a6ba-4c29-b88b-8d38c1b08840 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:42.445951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af943bbf-a6ba-4c29-b88b-8d38c1b08840 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:42.456176 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1095/2186] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:42 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:09:43.823425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1089bd-9a0c-4891-99ad-2bf9cc3471a2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:43.823956 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1089bd-9a0c-4891-99ad-2bf9cc3471a2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:43.824052 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1089bd-9a0c-4891-99ad-2bf9cc3471a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:43.841996 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8f1089bd-9a0c-4891-99ad-2bf9cc3471a2 None None] MFA Rules not processed for user `2b909d4c7d9b45b29f995268f8fac987`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:43.843114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8f1089bd-9a0c-4891-99ad-2bf9cc3471a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:43.854333 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1092/2187] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:43 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:44.405156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fc3af-d0ec-4207-a76e-837eeecb841f None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:44.406251 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fc3af-d0ec-4207-a76e-837eeecb841f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:44.406849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323fc3af-d0ec-4207-a76e-837eeecb841f None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:44.408517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1096/2188] 10.222.0.85 () {54 vars in 921 bytes} [Wed May 25 16:09:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:09:44.417503 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9ab7a-c6b5-48cf-a1a6-60983204b1de None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:44.418459 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9ab7a-c6b5-48cf-a1a6-60983204b1de None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:44.418459 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca9ab7a-c6b5-48cf-a1a6-60983204b1de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:44.436360 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-2ca9ab7a-c6b5-48cf-a1a6-60983204b1de None None] MFA Rules not processed for user `82b738f1b8c64590955a640c461b4ef9`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:44.437661 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2ca9ab7a-c6b5-48cf-a1a6-60983204b1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:44.520932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1093/2189] 10.222.0.85 () {58 vars in 1018 bytes} [Wed May 25 16:09:44 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:44.533786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323fc3af-d0ec-4207-a76e-837eeecb841f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:44.539605 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-323fc3af-d0ec-4207-a76e-837eeecb841f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:44.557792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323fc3af-d0ec-4207-a76e-837eeecb841f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:44.568815 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c176c-bf83-4c6c-889d-de229ca1772d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b738f1b8c64590955a640c461b4ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:44.569952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c176c-bf83-4c6c-889d-de229ca1772d service placement] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:44.570066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c176c-bf83-4c6c-889d-de229ca1772d service placement] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:44.570391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-474c176c-bf83-4c6c-889d-de229ca1772d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:44.580548 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c176c-bf83-4c6c-889d-de229ca1772d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:44.582192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-474c176c-bf83-4c6c-889d-de229ca1772d service placement] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:44.587767 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:44.596811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1097/2190] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:44 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:09:45.038348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9ab7a-c6b5-48cf-a1a6-60983204b1de None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:45.039723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2ca9ab7a-c6b5-48cf-a1a6-60983204b1de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:45.114394 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca9ab7a-c6b5-48cf-a1a6-60983204b1de None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:45.119192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08616a-2c3f-46e1-9577-34bca16dfdaf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:45.120215 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08616a-2c3f-46e1-9577-34bca16dfdaf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:45.120513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08616a-2c3f-46e1-9577-34bca16dfdaf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:45.120806 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff08616a-2c3f-46e1-9577-34bca16dfdaf admin admin] PATH_INFO: `/v3/users/b99619a058ce4e9d9c014a45bac91e07` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:45.122102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08616a-2c3f-46e1-9577-34bca16dfdaf admin admin] RBAC: Authorizing `identity:delete_user(user_id=b99619a058ce4e9d9c014a45bac91e07)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:45.124121 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff08616a-2c3f-46e1-9577-34bca16dfdaf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:45.137308 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:45.174909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ff08616a-2c3f-46e1-9577-34bca16dfdaf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b99619a058ce4e9d9c014a45bac91e07'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:45.189639 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ff08616a-2c3f-46e1-9577-34bca16dfdaf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b99619a058ce4e9d9c014a45bac91e07'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:45.198396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ff08616a-2c3f-46e1-9577-34bca16dfdaf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b99619a058ce4e9d9c014a45bac91e07'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:45.199019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ff08616a-2c3f-46e1-9577-34bca16dfdaf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b99619a058ce4e9d9c014a45bac91e07'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:45.206421 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ff08616a-2c3f-46e1-9577-34bca16dfdaf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b99619a058ce4e9d9c014a45bac91e07'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:45.212501 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1094/2191] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:09:45 2022] DELETE /identity/v3/users/b99619a058ce4e9d9c014a45bac91e07 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:45.420035 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c176c-bf83-4c6c-889d-de229ca1772d service placement] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:45.433669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-474c176c-bf83-4c6c-889d-de229ca1772d service placement] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:45.506002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb9770-5c2e-4078-ae23-a44fccb76ac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:45.506956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb9770-5c2e-4078-ae23-a44fccb76ac7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:45.507248 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb9770-5c2e-4078-ae23-a44fccb76ac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:45.507536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01bb9770-5c2e-4078-ae23-a44fccb76ac7 admin admin] PATH_INFO: `/v3/projects/f7f047df998e495798bf2660814db661` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:45.508870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb9770-5c2e-4078-ae23-a44fccb76ac7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7f047df998e495798bf2660814db661)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:45.510828 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01bb9770-5c2e-4078-ae23-a44fccb76ac7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:45.572737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-01bb9770-5c2e-4078-ae23-a44fccb76ac7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7f047df998e495798bf2660814db661', 'request_id': 'req-01bb9770-5c2e-4078-ae23-a44fccb76ac7'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:45.579526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-01bb9770-5c2e-4078-ae23-a44fccb76ac7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7f047df998e495798bf2660814db661', 'request_id': 'req-01bb9770-5c2e-4078-ae23-a44fccb76ac7'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:45.588216 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-01bb9770-5c2e-4078-ae23-a44fccb76ac7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7f047df998e495798bf2660814db661', 'request_id': 'req-01bb9770-5c2e-4078-ae23-a44fccb76ac7'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:45.601039 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-01bb9770-5c2e-4078-ae23-a44fccb76ac7 admin admin] The token cache is being invalidate because project f7f047df998e495798bf2660814db661 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:09:45.601255 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-01bb9770-5c2e-4078-ae23-a44fccb76ac7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:45.601985 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-01bb9770-5c2e-4078-ae23-a44fccb76ac7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:45.603673 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1098/2192] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:09:45 2022] DELETE /identity/v3/projects/f7f047df998e495798bf2660814db661 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:45.607887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08616a-2c3f-46e1-9577-34bca16dfdaf admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:45.609344 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ff08616a-2c3f-46e1-9577-34bca16dfdaf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:45.622499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff08616a-2c3f-46e1-9577-34bca16dfdaf admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:45.689470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68a394e-8a1d-4a74-a4cc-119a634bcfed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:45.690239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a394e-8a1d-4a74-a4cc-119a634bcfed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:45.690524 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a394e-8a1d-4a74-a4cc-119a634bcfed admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:45.690803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68a394e-8a1d-4a74-a4cc-119a634bcfed admin admin] PATH_INFO: `/v3/users/10e6196f4293496ba39524b3aa881169` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:45.692091 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a394e-8a1d-4a74-a4cc-119a634bcfed admin admin] RBAC: Authorizing `identity:delete_user(user_id=10e6196f4293496ba39524b3aa881169)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:45.694034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b68a394e-8a1d-4a74-a4cc-119a634bcfed admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:45.747751 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b68a394e-8a1d-4a74-a4cc-119a634bcfed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '10e6196f4293496ba39524b3aa881169'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:45.762965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b68a394e-8a1d-4a74-a4cc-119a634bcfed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '10e6196f4293496ba39524b3aa881169'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:45.772060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b68a394e-8a1d-4a74-a4cc-119a634bcfed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '10e6196f4293496ba39524b3aa881169'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:45.772758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b68a394e-8a1d-4a74-a4cc-119a634bcfed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '10e6196f4293496ba39524b3aa881169'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:45.781070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b68a394e-8a1d-4a74-a4cc-119a634bcfed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '10e6196f4293496ba39524b3aa881169'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:45.787605 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1095/2193] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:09:45 2022] DELETE /identity/v3/users/10e6196f4293496ba39524b3aa881169 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:45.995205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb9770-5c2e-4078-ae23-a44fccb76ac7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:45.997131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-01bb9770-5c2e-4078-ae23-a44fccb76ac7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:46.012598 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01bb9770-5c2e-4078-ae23-a44fccb76ac7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:46.079846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-713ff58a-11c9-4e6c-ab8b-993cbdeab855 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:46.080746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ff58a-11c9-4e6c-ab8b-993cbdeab855 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:46.081033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ff58a-11c9-4e6c-ab8b-993cbdeab855 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:46.081317 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-713ff58a-11c9-4e6c-ab8b-993cbdeab855 admin admin] PATH_INFO: `/v3/projects/d97854d072824236bfa880a44fa3aced` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:46.082621 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ff58a-11c9-4e6c-ab8b-993cbdeab855 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d97854d072824236bfa880a44fa3aced)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:46.084603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-713ff58a-11c9-4e6c-ab8b-993cbdeab855 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:46.127761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:46.148963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-713ff58a-11c9-4e6c-ab8b-993cbdeab855 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd97854d072824236bfa880a44fa3aced', 'request_id': 'req-713ff58a-11c9-4e6c-ab8b-993cbdeab855'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:46.156312 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-713ff58a-11c9-4e6c-ab8b-993cbdeab855 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd97854d072824236bfa880a44fa3aced', 'request_id': 'req-713ff58a-11c9-4e6c-ab8b-993cbdeab855'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:46.166240 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-713ff58a-11c9-4e6c-ab8b-993cbdeab855 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd97854d072824236bfa880a44fa3aced', 'request_id': 'req-713ff58a-11c9-4e6c-ab8b-993cbdeab855'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:46.177616 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-713ff58a-11c9-4e6c-ab8b-993cbdeab855 admin admin] The token cache is being invalidate because project d97854d072824236bfa880a44fa3aced was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:09:46.177912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-713ff58a-11c9-4e6c-ab8b-993cbdeab855 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:46.178552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-713ff58a-11c9-4e6c-ab8b-993cbdeab855 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:46.180287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1099/2194] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:09:45 2022] DELETE /identity/v3/projects/d97854d072824236bfa880a44fa3aced => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:46.185187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1a6c4-1715-4dde-a163-826cad7538ad None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:46.185676 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1a6c4-1715-4dde-a163-826cad7538ad None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:46.185766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f1a6c4-1715-4dde-a163-826cad7538ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:46.204294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d0f1a6c4-1715-4dde-a163-826cad7538ad None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:46.205539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d0f1a6c4-1715-4dde-a163-826cad7538ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:46.210448 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:46.276888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1096/2195] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:46.285877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1648ad8-abe9-4d46-8e13-f9c05e6e577b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:46.286171 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1648ad8-abe9-4d46-8e13-f9c05e6e577b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:46.286610 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1648ad8-abe9-4d46-8e13-f9c05e6e577b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:46.314016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e1648ad8-abe9-4d46-8e13-f9c05e6e577b None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:46.315456 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e1648ad8-abe9-4d46-8e13-f9c05e6e577b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:46.324087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1100/2196] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:46.328533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1a6c4-1715-4dde-a163-826cad7538ad None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:46.338551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f1a6c4-1715-4dde-a163-826cad7538ad None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:46.343193 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3c164-451f-4f44-8907-3bcdb4de12e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:46.343914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3c164-451f-4f44-8907-3bcdb4de12e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:46.344208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3c164-451f-4f44-8907-3bcdb4de12e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:46.344497 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d3c164-451f-4f44-8907-3bcdb4de12e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:46.345253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3c164-451f-4f44-8907-3bcdb4de12e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:46.345570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3c164-451f-4f44-8907-3bcdb4de12e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:46.347198 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d3c164-451f-4f44-8907-3bcdb4de12e1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:46.359331 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1097/2197] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:09:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:09:46.363798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1648ad8-abe9-4d46-8e13-f9c05e6e577b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:46.373548 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1648ad8-abe9-4d46-8e13-f9c05e6e577b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:46.378490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b997b8-9095-4804-aabb-9141efa014d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:46.379156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b997b8-9095-4804-aabb-9141efa014d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:46.379444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b997b8-9095-4804-aabb-9141efa014d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:46.379723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b997b8-9095-4804-aabb-9141efa014d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:46.380511 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b997b8-9095-4804-aabb-9141efa014d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:46.383594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b997b8-9095-4804-aabb-9141efa014d0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:46.417961 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1101/2198] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:09:46 2022] POST /identity/v3/projects => generated 383 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:46.421178 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3c164-451f-4f44-8907-3bcdb4de12e1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:46.432363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d3c164-451f-4f44-8907-3bcdb4de12e1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:46.492427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9d62c-4cf4-4ff3-b561-14b7a8532b2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:46.493086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9d62c-4cf4-4ff3-b561-14b7a8532b2a admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:46.493373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9d62c-4cf4-4ff3-b561-14b7a8532b2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:46.493658 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff9d62c-4cf4-4ff3-b561-14b7a8532b2a admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:46.494374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9d62c-4cf4-4ff3-b561-14b7a8532b2a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:46.496355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff9d62c-4cf4-4ff3-b561-14b7a8532b2a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:46.519544 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1098/2199] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:09:46 2022] POST /identity/v3/users => generated 292 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:46.524610 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b997b8-9095-4804-aabb-9141efa014d0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:46.536020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b997b8-9095-4804-aabb-9141efa014d0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:46.541517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108fd77-912b-47a6-895b-79f6a3c743c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:46.542193 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108fd77-912b-47a6-895b-79f6a3c743c9 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:46.542486 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108fd77-912b-47a6-895b-79f6a3c743c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:46.542769 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8108fd77-912b-47a6-895b-79f6a3c743c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:46.543514 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108fd77-912b-47a6-895b-79f6a3c743c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:46.545207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8108fd77-912b-47a6-895b-79f6a3c743c9 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:46.559267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1102/2200] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:46.562337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9d62c-4cf4-4ff3-b561-14b7a8532b2a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:46.574966 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff9d62c-4cf4-4ff3-b561-14b7a8532b2a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:46.579923 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92d74e4f-7cc5-474d-87a1-5381a5732adf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:46.580599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d74e4f-7cc5-474d-87a1-5381a5732adf admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:46.580889 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d74e4f-7cc5-474d-87a1-5381a5732adf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:46.581177 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92d74e4f-7cc5-474d-87a1-5381a5732adf admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:46.582007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d74e4f-7cc5-474d-87a1-5381a5732adf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:46.583571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92d74e4f-7cc5-474d-87a1-5381a5732adf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:46.600160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1099/2201] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:46.603290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108fd77-912b-47a6-895b-79f6a3c743c9 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:46.614524 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8108fd77-912b-47a6-895b-79f6a3c743c9 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:46.619692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d6d75-5dd8-4eae-90d1-4dbe4804266d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:46.620740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d6d75-5dd8-4eae-90d1-4dbe4804266d admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:46.621035 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d6d75-5dd8-4eae-90d1-4dbe4804266d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:46.621349 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8d6d75-5dd8-4eae-90d1-4dbe4804266d admin admin] PATH_INFO: `/v3/projects/44b1c124d7fa464c8b63523b02393fcd/users/949ca1d74310456bb0cf65d7d2a3416d/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:46.656422 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d6d75-5dd8-4eae-90d1-4dbe4804266d admin admin] RBAC: Authorizing `identity:create_grant(project_id=44b1c124d7fa464c8b63523b02393fcd, user_id=949ca1d74310456bb0cf65d7d2a3416d, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:46.660254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8d6d75-5dd8-4eae-90d1-4dbe4804266d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:46.678366 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1103/2202] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:09:46 2022] PUT /identity/v3/projects/44b1c124d7fa464c8b63523b02393fcd/users/949ca1d74310456bb0cf65d7d2a3416d/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:46.689039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527fd62-2648-4292-b7b1-44e721f44c85 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:46.689322 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527fd62-2648-4292-b7b1-44e721f44c85 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:46.689702 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0527fd62-2648-4292-b7b1-44e721f44c85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:46.730372 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0527fd62-2648-4292-b7b1-44e721f44c85 None None] MFA Rules not processed for user `949ca1d74310456bb0cf65d7d2a3416d`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:46.731624 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0527fd62-2648-4292-b7b1-44e721f44c85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:46.810170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1100/2203] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:46 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:46.814700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d6d75-5dd8-4eae-90d1-4dbe4804266d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:46.878609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8d6d75-5dd8-4eae-90d1-4dbe4804266d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:46.884323 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01036d9a-0320-4f08-9504-e6bf06a7454d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:46.884991 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01036d9a-0320-4f08-9504-e6bf06a7454d admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:46.885280 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01036d9a-0320-4f08-9504-e6bf06a7454d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:46.885570 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01036d9a-0320-4f08-9504-e6bf06a7454d admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:46.886266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01036d9a-0320-4f08-9504-e6bf06a7454d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:46.888238 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01036d9a-0320-4f08-9504-e6bf06a7454d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:46.916233 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1104/2204] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:09:46 2022] POST /identity/v3/projects => generated 383 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:46.920416 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527fd62-2648-4292-b7b1-44e721f44c85 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:46.927128 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:46.932107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0527fd62-2648-4292-b7b1-44e721f44c85 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:46.990429 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb654c9-ec59-4736-8e8a-ca2d4123c39f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:46.991103 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb654c9-ec59-4736-8e8a-ca2d4123c39f admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:46.991390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb654c9-ec59-4736-8e8a-ca2d4123c39f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:46.991671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb654c9-ec59-4736-8e8a-ca2d4123c39f admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:46.992503 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb654c9-ec59-4736-8e8a-ca2d4123c39f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:46.994458 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb654c9-ec59-4736-8e8a-ca2d4123c39f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:47.016250 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1101/2205] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:09:46 2022] POST /identity/v3/users => generated 298 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:09:47.020323 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01036d9a-0320-4f08-9504-e6bf06a7454d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:47.032033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01036d9a-0320-4f08-9504-e6bf06a7454d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:47.040291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47371fa0-d9be-47a5-bd47-81dcedc97097 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:47.041480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47371fa0-d9be-47a5-bd47-81dcedc97097 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:47.042002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47371fa0-d9be-47a5-bd47-81dcedc97097 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:47.042509 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47371fa0-d9be-47a5-bd47-81dcedc97097 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:47.043966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47371fa0-d9be-47a5-bd47-81dcedc97097 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:47.047079 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47371fa0-d9be-47a5-bd47-81dcedc97097 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:47.059161 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1105/2206] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:47.064055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb654c9-ec59-4736-8e8a-ca2d4123c39f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:47.080470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb654c9-ec59-4736-8e8a-ca2d4123c39f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:47.087657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559b547-25b2-4ea8-af59-d5cac6bfb51f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:47.088771 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559b547-25b2-4ea8-af59-d5cac6bfb51f admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:47.088895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559b547-25b2-4ea8-af59-d5cac6bfb51f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:47.089172 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6559b547-25b2-4ea8-af59-d5cac6bfb51f admin admin] PATH_INFO: `/v3/projects/e9d790283c8648fa8b1cc8736538531e/users/fe0ba91031d74ee381e74abf64085a05/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:47.105401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559b547-25b2-4ea8-af59-d5cac6bfb51f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9d790283c8648fa8b1cc8736538531e, user_id=fe0ba91031d74ee381e74abf64085a05, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:47.108110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6559b547-25b2-4ea8-af59-d5cac6bfb51f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:47.122004 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1102/2207] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:09:47 2022] PUT /identity/v3/projects/e9d790283c8648fa8b1cc8736538531e/users/fe0ba91031d74ee381e74abf64085a05/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:47.125937 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47371fa0-d9be-47a5-bd47-81dcedc97097 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:47.135966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47371fa0-d9be-47a5-bd47-81dcedc97097 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:47.168229 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:47.202837 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeb7cee-bc34-4dbe-ab1b-7ccd4ffaa327 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:47.203528 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb7cee-bc34-4dbe-ab1b-7ccd4ffaa327 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:47.203842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb7cee-bc34-4dbe-ab1b-7ccd4ffaa327 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:47.204144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeb7cee-bc34-4dbe-ab1b-7ccd4ffaa327 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:47.204883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb7cee-bc34-4dbe-ab1b-7ccd4ffaa327 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:47.206511 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeb7cee-bc34-4dbe-ab1b-7ccd4ffaa327 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:47.220613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1106/2208] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:47 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:47.224694 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559b547-25b2-4ea8-af59-d5cac6bfb51f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:47.236982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6559b547-25b2-4ea8-af59-d5cac6bfb51f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:47.241909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532ce30-657b-4596-9222-7ef17cadc6ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:47.242917 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532ce30-657b-4596-9222-7ef17cadc6ec admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:47.243445 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532ce30-657b-4596-9222-7ef17cadc6ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:47.243800 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5532ce30-657b-4596-9222-7ef17cadc6ec admin admin] PATH_INFO: `/v3/projects/e9d790283c8648fa8b1cc8736538531e/users/fe0ba91031d74ee381e74abf64085a05/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:47.246136 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532ce30-657b-4596-9222-7ef17cadc6ec admin admin] RBAC: Authorizing `identity:create_grant(project_id=e9d790283c8648fa8b1cc8736538531e, user_id=fe0ba91031d74ee381e74abf64085a05, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:47.248679 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5532ce30-657b-4596-9222-7ef17cadc6ec admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:47.268176 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1103/2209] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:09:47 2022] PUT /identity/v3/projects/e9d790283c8648fa8b1cc8736538531e/users/fe0ba91031d74ee381e74abf64085a05/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:47.278582 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7cb6d-d5d7-4121-8f62-f895511cd27e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:47.279337 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7cb6d-d5d7-4121-8f62-f895511cd27e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:47.279337 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f7cb6d-d5d7-4121-8f62-f895511cd27e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:47.309537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-56f7cb6d-d5d7-4121-8f62-f895511cd27e None None] MFA Rules not processed for user `fe0ba91031d74ee381e74abf64085a05`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:47.311084 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-56f7cb6d-d5d7-4121-8f62-f895511cd27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:47.395480 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1107/2210] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:47 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:09:47.404173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532ce30-657b-4596-9222-7ef17cadc6ec admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:47.407492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5532ce30-657b-4596-9222-7ef17cadc6ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:47.470707 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5532ce30-657b-4596-9222-7ef17cadc6ec admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:47.474701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c28c375-3bde-487d-bf2d-2a0e0f5e5da4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:47.475354 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28c375-3bde-487d-bf2d-2a0e0f5e5da4 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:47.475645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28c375-3bde-487d-bf2d-2a0e0f5e5da4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:47.476030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c28c375-3bde-487d-bf2d-2a0e0f5e5da4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:47.535657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28c375-3bde-487d-bf2d-2a0e0f5e5da4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:47.537302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c28c375-3bde-487d-bf2d-2a0e0f5e5da4 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:47.547100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1104/2211] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:47 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:09:49.933152 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7cb6d-d5d7-4121-8f62-f895511cd27e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:49.934887 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-56f7cb6d-d5d7-4121-8f62-f895511cd27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:50.014284 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f7cb6d-d5d7-4121-8f62-f895511cd27e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:50.019204 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cefab-3cd0-403f-9cd0-6c0ddea3691c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:50.020136 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cefab-3cd0-403f-9cd0-6c0ddea3691c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:50.020431 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cefab-3cd0-403f-9cd0-6c0ddea3691c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:50.020712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8cefab-3cd0-403f-9cd0-6c0ddea3691c admin admin] PATH_INFO: `/v3/users/0decd125ebb448f5a5c92ae3212b2f70` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:50.022050 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cefab-3cd0-403f-9cd0-6c0ddea3691c admin admin] RBAC: Authorizing `identity:delete_user(user_id=0decd125ebb448f5a5c92ae3212b2f70)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:50.024378 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac8cefab-3cd0-403f-9cd0-6c0ddea3691c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:50.081400 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ac8cefab-3cd0-403f-9cd0-6c0ddea3691c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0decd125ebb448f5a5c92ae3212b2f70'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:50.082192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ac8cefab-3cd0-403f-9cd0-6c0ddea3691c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0decd125ebb448f5a5c92ae3212b2f70'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:50.087682 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:50.096486 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ac8cefab-3cd0-403f-9cd0-6c0ddea3691c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0decd125ebb448f5a5c92ae3212b2f70'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:50.104663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ac8cefab-3cd0-403f-9cd0-6c0ddea3691c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0decd125ebb448f5a5c92ae3212b2f70'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:50.105341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ac8cefab-3cd0-403f-9cd0-6c0ddea3691c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0decd125ebb448f5a5c92ae3212b2f70'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:50.117986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1108/2212] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:09:49 2022] DELETE /identity/v3/users/0decd125ebb448f5a5c92ae3212b2f70 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:50.122507 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a88a59-d2db-49ee-9469-c3867bb4e192 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:50.122963 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a88a59-d2db-49ee-9469-c3867bb4e192 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:50.123086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a88a59-d2db-49ee-9469-c3867bb4e192 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:50.140618 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-73a88a59-d2db-49ee-9469-c3867bb4e192 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:50.141956 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-73a88a59-d2db-49ee-9469-c3867bb4e192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:50.148310 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:50.207212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1105/2213] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:50.217017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cefab-3cd0-403f-9cd0-6c0ddea3691c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:50.218439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ac8cefab-3cd0-403f-9cd0-6c0ddea3691c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:50.281070 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac8cefab-3cd0-403f-9cd0-6c0ddea3691c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:50.284700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a57141-4477-43e2-83c5-668a79065b46 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:50.285370 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a57141-4477-43e2-83c5-668a79065b46 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:50.285663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a57141-4477-43e2-83c5-668a79065b46 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:50.285956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56a57141-4477-43e2-83c5-668a79065b46 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:50.295458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a57141-4477-43e2-83c5-668a79065b46 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:50.296946 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56a57141-4477-43e2-83c5-668a79065b46 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:50.311039 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1109/2214] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:09:50.633667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a88a59-d2db-49ee-9469-c3867bb4e192 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:50.635252 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-73a88a59-d2db-49ee-9469-c3867bb4e192 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:50.649752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a88a59-d2db-49ee-9469-c3867bb4e192 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:50.657547 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da8547f-f3de-4168-82a3-68fa2860b16b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:50.658490 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8547f-f3de-4168-82a3-68fa2860b16b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:50.658773 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8547f-f3de-4168-82a3-68fa2860b16b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:50.659057 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da8547f-f3de-4168-82a3-68fa2860b16b admin admin] PATH_INFO: `/v3/projects/7cec4cc428754579bcf1a56f555bc363` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:50.660880 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8547f-f3de-4168-82a3-68fa2860b16b admin admin] RBAC: Authorizing `identity:delete_project(project_id=7cec4cc428754579bcf1a56f555bc363)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:50.663132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da8547f-f3de-4168-82a3-68fa2860b16b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:50.719614 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5da8547f-f3de-4168-82a3-68fa2860b16b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7cec4cc428754579bcf1a56f555bc363', 'request_id': 'req-5da8547f-f3de-4168-82a3-68fa2860b16b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:50.734019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5da8547f-f3de-4168-82a3-68fa2860b16b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7cec4cc428754579bcf1a56f555bc363', 'request_id': 'req-5da8547f-f3de-4168-82a3-68fa2860b16b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:50.744230 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5da8547f-f3de-4168-82a3-68fa2860b16b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7cec4cc428754579bcf1a56f555bc363', 'request_id': 'req-5da8547f-f3de-4168-82a3-68fa2860b16b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:50.759232 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5da8547f-f3de-4168-82a3-68fa2860b16b admin admin] The token cache is being invalidate because project 7cec4cc428754579bcf1a56f555bc363 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:09:50.759544 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5da8547f-f3de-4168-82a3-68fa2860b16b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:50.761806 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5da8547f-f3de-4168-82a3-68fa2860b16b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:50.763698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1106/2215] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:09:50 2022] DELETE /identity/v3/projects/7cec4cc428754579bcf1a56f555bc363 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:51.882536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a57141-4477-43e2-83c5-668a79065b46 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:51.884055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-56a57141-4477-43e2-83c5-668a79065b46 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:51.942263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56a57141-4477-43e2-83c5-668a79065b46 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:52.000661 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0762f2-6f72-4ebd-aad0-feb1f641c898 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:52.001617 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0762f2-6f72-4ebd-aad0-feb1f641c898 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:52.001987 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0762f2-6f72-4ebd-aad0-feb1f641c898 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:52.002358 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e0762f2-6f72-4ebd-aad0-feb1f641c898 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:52.005653 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9e0762f2-6f72-4ebd-aad0-feb1f641c898 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:52.050684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:52.073700 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0762f2-6f72-4ebd-aad0-feb1f641c898 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:52.075165 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e0762f2-6f72-4ebd-aad0-feb1f641c898 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:52.084974 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1110/2216] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:51 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 203 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:09:54.225689 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6776ee6-46a6-4d3b-a09c-a676b772928c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:54.226312 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6776ee6-46a6-4d3b-a09c-a676b772928c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:54.226547 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6776ee6-46a6-4d3b-a09c-a676b772928c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:54.246393 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e6776ee6-46a6-4d3b-a09c-a676b772928c None None] MFA Rules not processed for user `d0427d9cce40423185a40df61c3617a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:54.247629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e6776ee6-46a6-4d3b-a09c-a676b772928c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:54.316393 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1107/2217] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:54 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:57.207523 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0762f2-6f72-4ebd-aad0-feb1f641c898 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:57.209193 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9e0762f2-6f72-4ebd-aad0-feb1f641c898 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:57.288396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e0762f2-6f72-4ebd-aad0-feb1f641c898 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:57.303423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7212085-60ae-4427-8a6f-c93747ad23c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:57.306975 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7212085-60ae-4427-8a6f-c93747ad23c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:57.309387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7212085-60ae-4427-8a6f-c93747ad23c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:57.309865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7212085-60ae-4427-8a6f-c93747ad23c9 admin admin] PATH_INFO: `/v3/users/949ca1d74310456bb0cf65d7d2a3416d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:57.311408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7212085-60ae-4427-8a6f-c93747ad23c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=949ca1d74310456bb0cf65d7d2a3416d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:57.319184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7212085-60ae-4427-8a6f-c93747ad23c9 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:57.403537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b7212085-60ae-4427-8a6f-c93747ad23c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '949ca1d74310456bb0cf65d7d2a3416d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:57.404401 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b7212085-60ae-4427-8a6f-c93747ad23c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '949ca1d74310456bb0cf65d7d2a3416d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:57.428419 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b7212085-60ae-4427-8a6f-c93747ad23c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '949ca1d74310456bb0cf65d7d2a3416d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:57.440751 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b7212085-60ae-4427-8a6f-c93747ad23c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '949ca1d74310456bb0cf65d7d2a3416d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:57.441625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b7212085-60ae-4427-8a6f-c93747ad23c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '949ca1d74310456bb0cf65d7d2a3416d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:57.466476 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1111/2218] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:09:57 2022] DELETE /identity/v3/users/949ca1d74310456bb0cf65d7d2a3416d => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:57.467654 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a7a00-be44-4171-9cd0-5f810dd7ef19 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:57.468876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a7a00-be44-4171-9cd0-5f810dd7ef19 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:57.469360 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62a7a00-be44-4171-9cd0-5f810dd7ef19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:57.502213 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b62a7a00-be44-4171-9cd0-5f810dd7ef19 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:57.503588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b62a7a00-be44-4171-9cd0-5f810dd7ef19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:57.565883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:57.597913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1108/2219] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:09:57.607810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7212085-60ae-4427-8a6f-c93747ad23c9 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:57.612400 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b7212085-60ae-4427-8a6f-c93747ad23c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:57.701004 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7212085-60ae-4427-8a6f-c93747ad23c9 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:57.704586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79292631-49d8-49b8-9ab8-f760afd37865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:57.705267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79292631-49d8-49b8-9ab8-f760afd37865 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:57.705557 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79292631-49d8-49b8-9ab8-f760afd37865 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:57.705837 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79292631-49d8-49b8-9ab8-f760afd37865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:57.715023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79292631-49d8-49b8-9ab8-f760afd37865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:57.716646 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79292631-49d8-49b8-9ab8-f760afd37865 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:57.721931 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:57.726956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1112/2220] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:09:58.160973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a7a00-be44-4171-9cd0-5f810dd7ef19 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:58.162930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b62a7a00-be44-4171-9cd0-5f810dd7ef19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:58.175061 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62a7a00-be44-4171-9cd0-5f810dd7ef19 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:58.192616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39312471-17c8-4e4f-a730-b2d4a89bb5eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:58.196526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39312471-17c8-4e4f-a730-b2d4a89bb5eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:58.196939 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39312471-17c8-4e4f-a730-b2d4a89bb5eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:58.197286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39312471-17c8-4e4f-a730-b2d4a89bb5eb admin admin] PATH_INFO: `/v3/projects/44b1c124d7fa464c8b63523b02393fcd` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:58.203405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39312471-17c8-4e4f-a730-b2d4a89bb5eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=44b1c124d7fa464c8b63523b02393fcd)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:58.205444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39312471-17c8-4e4f-a730-b2d4a89bb5eb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:58.259707 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-39312471-17c8-4e4f-a730-b2d4a89bb5eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '44b1c124d7fa464c8b63523b02393fcd', 'request_id': 'req-39312471-17c8-4e4f-a730-b2d4a89bb5eb'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:58.288542 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-39312471-17c8-4e4f-a730-b2d4a89bb5eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '44b1c124d7fa464c8b63523b02393fcd', 'request_id': 'req-39312471-17c8-4e4f-a730-b2d4a89bb5eb'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:58.303969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-39312471-17c8-4e4f-a730-b2d4a89bb5eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '44b1c124d7fa464c8b63523b02393fcd', 'request_id': 'req-39312471-17c8-4e4f-a730-b2d4a89bb5eb'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:58.318300 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-39312471-17c8-4e4f-a730-b2d4a89bb5eb admin admin] The token cache is being invalidate because project 44b1c124d7fa464c8b63523b02393fcd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:09:58.318516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-39312471-17c8-4e4f-a730-b2d4a89bb5eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:58.319325 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-39312471-17c8-4e4f-a730-b2d4a89bb5eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:58.321207 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1109/2221] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:09:58 2022] DELETE /identity/v3/projects/44b1c124d7fa464c8b63523b02393fcd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:58.325154 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79292631-49d8-49b8-9ab8-f760afd37865 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:58.335183 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79292631-49d8-49b8-9ab8-f760afd37865 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:58.396414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15db45f5-61cf-49cb-9c59-c85ed96da092 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:58.397326 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db45f5-61cf-49cb-9c59-c85ed96da092 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:58.397614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db45f5-61cf-49cb-9c59-c85ed96da092 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:58.397896 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15db45f5-61cf-49cb-9c59-c85ed96da092 admin admin] PATH_INFO: `/v3/users/fe0ba91031d74ee381e74abf64085a05` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:58.399462 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db45f5-61cf-49cb-9c59-c85ed96da092 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe0ba91031d74ee381e74abf64085a05)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:58.401463 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15db45f5-61cf-49cb-9c59-c85ed96da092 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:58.451215 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-15db45f5-61cf-49cb-9c59-c85ed96da092 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe0ba91031d74ee381e74abf64085a05'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:58.452060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-15db45f5-61cf-49cb-9c59-c85ed96da092 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe0ba91031d74ee381e74abf64085a05'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:58.464035 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-15db45f5-61cf-49cb-9c59-c85ed96da092 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe0ba91031d74ee381e74abf64085a05'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:58.473475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-15db45f5-61cf-49cb-9c59-c85ed96da092 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe0ba91031d74ee381e74abf64085a05'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:58.474157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-15db45f5-61cf-49cb-9c59-c85ed96da092 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe0ba91031d74ee381e74abf64085a05'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:58.486552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1113/2222] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:09:58 2022] DELETE /identity/v3/users/fe0ba91031d74ee381e74abf64085a05 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:59.038159 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39312471-17c8-4e4f-a730-b2d4a89bb5eb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:59.043198 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-39312471-17c8-4e4f-a730-b2d4a89bb5eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:59.077912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39312471-17c8-4e4f-a730-b2d4a89bb5eb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:59.173327 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58d08313-f70d-414b-8188-61cebf0e8921 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:59.174228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d08313-f70d-414b-8188-61cebf0e8921 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:59.174520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d08313-f70d-414b-8188-61cebf0e8921 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:59.174803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58d08313-f70d-414b-8188-61cebf0e8921 admin admin] PATH_INFO: `/v3/projects/e9d790283c8648fa8b1cc8736538531e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:59.176610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d08313-f70d-414b-8188-61cebf0e8921 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e9d790283c8648fa8b1cc8736538531e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:59.178577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58d08313-f70d-414b-8188-61cebf0e8921 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:59.232723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58d08313-f70d-414b-8188-61cebf0e8921 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e9d790283c8648fa8b1cc8736538531e', 'request_id': 'req-58d08313-f70d-414b-8188-61cebf0e8921'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:59.243143 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58d08313-f70d-414b-8188-61cebf0e8921 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e9d790283c8648fa8b1cc8736538531e', 'request_id': 'req-58d08313-f70d-414b-8188-61cebf0e8921'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:59.251700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58d08313-f70d-414b-8188-61cebf0e8921 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e9d790283c8648fa8b1cc8736538531e', 'request_id': 'req-58d08313-f70d-414b-8188-61cebf0e8921'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:59.257596 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:59.266669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58d08313-f70d-414b-8188-61cebf0e8921 admin admin] The token cache is being invalidate because project e9d790283c8648fa8b1cc8736538531e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:09:59.267770 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58d08313-f70d-414b-8188-61cebf0e8921 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:59.269127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-58d08313-f70d-414b-8188-61cebf0e8921 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:09:59.271643 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1110/2223] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:09:59 2022] DELETE /identity/v3/projects/e9d790283c8648fa8b1cc8736538531e => generated 0 bytes in 234 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:59.279336 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6839c-e7e4-40f9-af21-09347aadb477 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:59.279850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6839c-e7e4-40f9-af21-09347aadb477 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:59.280822 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb6839c-e7e4-40f9-af21-09347aadb477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:59.318125 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-adb6839c-e7e4-40f9-af21-09347aadb477 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:59.319581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-adb6839c-e7e4-40f9-af21-09347aadb477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:59.399711 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1114/2224] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:59.408265 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12679a4e-1d67-41d3-937b-a10c94e0b35d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:59.408568 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12679a4e-1d67-41d3-937b-a10c94e0b35d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:59.408858 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12679a4e-1d67-41d3-937b-a10c94e0b35d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:59.428682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-12679a4e-1d67-41d3-937b-a10c94e0b35d None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:59.429929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-12679a4e-1d67-41d3-937b-a10c94e0b35d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:59.438707 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1111/2225] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:09:59.444797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6839c-e7e4-40f9-af21-09347aadb477 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:59.458184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb6839c-e7e4-40f9-af21-09347aadb477 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:59.463116 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e793f-6230-4d65-b943-f1b656bd376e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:59.463923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e793f-6230-4d65-b943-f1b656bd376e admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:59.464231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e793f-6230-4d65-b943-f1b656bd376e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:59.464502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079e793f-6230-4d65-b943-f1b656bd376e admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:59.465304 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e793f-6230-4d65-b943-f1b656bd376e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:09:59.466305 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e793f-6230-4d65-b943-f1b656bd376e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:59.468052 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079e793f-6230-4d65-b943-f1b656bd376e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:59.475481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:09:59.483741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1115/2226] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:09:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:09:59.488221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12679a4e-1d67-41d3-937b-a10c94e0b35d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:59.499562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12679a4e-1d67-41d3-937b-a10c94e0b35d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:59.505253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84680ac-cd6c-4ac1-a379-2b53179b175b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:59.505927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84680ac-cd6c-4ac1-a379-2b53179b175b admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:59.506221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84680ac-cd6c-4ac1-a379-2b53179b175b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:59.506508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f84680ac-cd6c-4ac1-a379-2b53179b175b admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:59.507213 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84680ac-cd6c-4ac1-a379-2b53179b175b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:59.509233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f84680ac-cd6c-4ac1-a379-2b53179b175b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:59.539926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1112/2227] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:09:59 2022] POST /identity/v3/projects => generated 363 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:09:59.541158 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e793f-6230-4d65-b943-f1b656bd376e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:59.551606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079e793f-6230-4d65-b943-f1b656bd376e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:59.614077 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf5b4d-458a-4e3f-b6a3-fae6099b933b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:59.614840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf5b4d-458a-4e3f-b6a3-fae6099b933b admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:59.615199 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf5b4d-458a-4e3f-b6a3-fae6099b933b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:59.615485 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8cf5b4d-458a-4e3f-b6a3-fae6099b933b admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:59.616224 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf5b4d-458a-4e3f-b6a3-fae6099b933b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:59.618264 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8cf5b4d-458a-4e3f-b6a3-fae6099b933b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:59.640123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1116/2228] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:09:59 2022] POST /identity/v3/users => generated 282 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:09:59.644514 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84680ac-cd6c-4ac1-a379-2b53179b175b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:59.656302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f84680ac-cd6c-4ac1-a379-2b53179b175b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:59.662178 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2aaf9-3967-4637-b80a-cc5e29cf6086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:59.662846 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2aaf9-3967-4637-b80a-cc5e29cf6086 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:59.663141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2aaf9-3967-4637-b80a-cc5e29cf6086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:59.663426 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c2aaf9-3967-4637-b80a-cc5e29cf6086 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:59.664223 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2aaf9-3967-4637-b80a-cc5e29cf6086 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:59.665877 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c2aaf9-3967-4637-b80a-cc5e29cf6086 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:59.677360 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1113/2229] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:59.681708 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf5b4d-458a-4e3f-b6a3-fae6099b933b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:59.692697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8cf5b4d-458a-4e3f-b6a3-fae6099b933b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:59.697558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cad5575-cb8b-431d-90f8-9947a31c2734 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:59.698226 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad5575-cb8b-431d-90f8-9947a31c2734 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:59.698511 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad5575-cb8b-431d-90f8-9947a31c2734 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:59.698790 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cad5575-cb8b-431d-90f8-9947a31c2734 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:59.699523 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad5575-cb8b-431d-90f8-9947a31c2734 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:59.701179 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cad5575-cb8b-431d-90f8-9947a31c2734 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:59.711941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1117/2230] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:09:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:09:59.716122 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2aaf9-3967-4637-b80a-cc5e29cf6086 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:59.726684 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c2aaf9-3967-4637-b80a-cc5e29cf6086 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:59.732427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed42757-222a-4de8-a397-92d1d1fe1d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:59.733359 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed42757-222a-4de8-a397-92d1d1fe1d06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:59.733646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed42757-222a-4de8-a397-92d1d1fe1d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:59.733939 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed42757-222a-4de8-a397-92d1d1fe1d06 admin admin] PATH_INFO: `/v3/projects/9f70c7a68d8e44b981b62fba5a517703/users/64110f2238ff42948c0e075afe014b74/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:59.750977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed42757-222a-4de8-a397-92d1d1fe1d06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f70c7a68d8e44b981b62fba5a517703, user_id=64110f2238ff42948c0e075afe014b74, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:59.753691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed42757-222a-4de8-a397-92d1d1fe1d06 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:59.770476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1114/2231] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:09:59 2022] PUT /identity/v3/projects/9f70c7a68d8e44b981b62fba5a517703/users/64110f2238ff42948c0e075afe014b74/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:09:59.782211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9567212-61a3-48ba-b889-62c41acdaeda None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:59.782558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9567212-61a3-48ba-b889-62c41acdaeda None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:59.782855 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9567212-61a3-48ba-b889-62c41acdaeda None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:59.813271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f9567212-61a3-48ba-b889-62c41acdaeda None None] MFA Rules not processed for user `64110f2238ff42948c0e075afe014b74`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:09:59.814506 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f9567212-61a3-48ba-b889-62c41acdaeda None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:59.882372 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1118/2232] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:09:59 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:09:59.894765 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed42757-222a-4de8-a397-92d1d1fe1d06 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:09:59.896777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6ed42757-222a-4de8-a397-92d1d1fe1d06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:09:59.960299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed42757-222a-4de8-a397-92d1d1fe1d06 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:09:59.965038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-300c492c-9c87-429a-8f9d-f721592a2f90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:09:59.965693 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c492c-9c87-429a-8f9d-f721592a2f90 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:09:59.965980 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c492c-9c87-429a-8f9d-f721592a2f90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:09:59.966262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-300c492c-9c87-429a-8f9d-f721592a2f90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:09:59.975068 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c492c-9c87-429a-8f9d-f721592a2f90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:09:59.976546 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-300c492c-9c87-429a-8f9d-f721592a2f90 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:09:59.986245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1115/2233] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:09:59 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:10:04.008292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a538c8d-d741-4555-96fb-363f59f10e82 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:04.008784 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a538c8d-d741-4555-96fb-363f59f10e82 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:04.008899 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a538c8d-d741-4555-96fb-363f59f10e82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:04.026517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8a538c8d-d741-4555-96fb-363f59f10e82 None None] MFA Rules not processed for user `708fdc537e1747d7bd5917ed7af9c368`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:04.027694 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8a538c8d-d741-4555-96fb-363f59f10e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:04.086552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1119/2234] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:10:04 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:10:09.982288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=1,keystone:SELECT=55,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:14.080848 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:16.412284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fd8b4-9b65-4b6a-97b0-deedd473caab None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:16.412745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fd8b4-9b65-4b6a-97b0-deedd473caab None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:16.412861 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948fd8b4-9b65-4b6a-97b0-deedd473caab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:16.444854 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-948fd8b4-9b65-4b6a-97b0-deedd473caab None None] MFA Rules not processed for user `066de1f11f724eee9df0377beb58d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:16.446244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-948fd8b4-9b65-4b6a-97b0-deedd473caab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:16.523745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1116/2235] 10.222.0.85 () {58 vars in 1009 bytes} [Wed May 25 16:10:16 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:10:16.581802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a538c8d-d741-4555-96fb-363f59f10e82 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:16.595222 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8a538c8d-d741-4555-96fb-363f59f10e82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:16.673126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a538c8d-d741-4555-96fb-363f59f10e82 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:16.686575 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ca7f9ce-0a7f-47b3-9266-286cdb2ef4e3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77fda27641c84fccb68a5cdaf4e8dc4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:16.687268 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7f9ce-0a7f-47b3-9266-286cdb2ef4e3 service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:16.687649 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7f9ce-0a7f-47b3-9266-286cdb2ef4e3 service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:16.687974 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ca7f9ce-0a7f-47b3-9266-286cdb2ef4e3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:16.696946 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7f9ce-0a7f-47b3-9266-286cdb2ef4e3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:16.698559 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ca7f9ce-0a7f-47b3-9266-286cdb2ef4e3 service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:16.711037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1120/2236] 10.222.0.85 () {60 vars in 1384 bytes} [Wed May 25 16:10:16 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:10:26.514147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=32 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:26.704023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=32 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:33.439182 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ce1d7-ce79-4ceb-8ccb-22e98d0871c7 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:33.439739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ce1d7-ce79-4ceb-8ccb-22e98d0871c7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:33.439848 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-732ce1d7-ce79-4ceb-8ccb-22e98d0871c7 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:33.441275 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1117/2237] 10.222.0.85 () {54 vars in 921 bytes} [Wed May 25 16:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:10:33.447692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e20b9-4729-480b-b077-41a833cc7481 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:33.448236 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e20b9-4729-480b-b077-41a833cc7481 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:33.448332 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648e20b9-4729-480b-b077-41a833cc7481 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:33.497234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-648e20b9-4729-480b-b077-41a833cc7481 None None] MFA Rules not processed for user `4751abc9f67947b3ad17ba2b78ce2c12`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:33.498483 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-648e20b9-4729-480b-b077-41a833cc7481 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:33.579609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1121/2238] 10.222.0.85 () {58 vars in 1018 bytes} [Wed May 25 16:10:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:10:33.591587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ce1d7-ce79-4ceb-8ccb-22e98d0871c7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:33.593485 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-732ce1d7-ce79-4ceb-8ccb-22e98d0871c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:33.676637 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-732ce1d7-ce79-4ceb-8ccb-22e98d0871c7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:33.680280 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7154436b-b9cf-46d6-98ad-37ac2af17c61 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '196dc406a95444c39babc50d59f9eb6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:33.680938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154436b-b9cf-46d6-98ad-37ac2af17c61 service glance] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:33.681222 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154436b-b9cf-46d6-98ad-37ac2af17c61 service glance] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:33.681526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7154436b-b9cf-46d6-98ad-37ac2af17c61 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:33.693329 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154436b-b9cf-46d6-98ad-37ac2af17c61 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:33.694788 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7154436b-b9cf-46d6-98ad-37ac2af17c61 service glance] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:33.706185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1118/2239] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:10:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:10:33.779501 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478088eb-1b3d-4669-83d2-5a3fb6b7a04a None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:33.779753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478088eb-1b3d-4669-83d2-5a3fb6b7a04a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:33.780062 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-478088eb-1b3d-4669-83d2-5a3fb6b7a04a None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:33.781449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1122/2240] 10.222.0.85 () {54 vars in 921 bytes} [Wed May 25 16:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:10:33.786457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447899ab-fe93-4fa6-a277-79eed462cfae None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:33.786731 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447899ab-fe93-4fa6-a277-79eed462cfae None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:33.786983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447899ab-fe93-4fa6-a277-79eed462cfae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:33.806496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-447899ab-fe93-4fa6-a277-79eed462cfae None None] MFA Rules not processed for user `413b3d65427b41dc99bc7b0a94fb1dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:33.807749 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-447899ab-fe93-4fa6-a277-79eed462cfae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:33.860461 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1119/2241] 10.222.0.85 () {58 vars in 1018 bytes} [Wed May 25 16:10:33 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:10:33.872736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478088eb-1b3d-4669-83d2-5a3fb6b7a04a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:33.874542 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-478088eb-1b3d-4669-83d2-5a3fb6b7a04a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:33.888809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-478088eb-1b3d-4669-83d2-5a3fb6b7a04a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:33.893211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd74b22-bf85-4360-a8a6-0e897bd2b9f6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:33.893894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd74b22-bf85-4360-a8a6-0e897bd2b9f6 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:33.894205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd74b22-bf85-4360-a8a6-0e897bd2b9f6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:33.894490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd74b22-bf85-4360-a8a6-0e897bd2b9f6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:33.902580 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd74b22-bf85-4360-a8a6-0e897bd2b9f6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:33.904076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd74b22-bf85-4360-a8a6-0e897bd2b9f6 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:33.912130 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1123/2242] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:10:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:10:33.997890 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1621b11-1dbc-4a08-9b1b-418b71f2d267 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:33.998189 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1621b11-1dbc-4a08-9b1b-418b71f2d267 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:33.998476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1621b11-1dbc-4a08-9b1b-418b71f2d267 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:33.999873 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1120/2243] 10.222.0.85 () {54 vars in 921 bytes} [Wed May 25 16:10:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:10:34.005194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ca570-04be-41cb-a618-9d72d1b57cbd None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:34.005490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ca570-04be-41cb-a618-9d72d1b57cbd None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:34.005780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d32ca570-04be-41cb-a618-9d72d1b57cbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:34.025650 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d32ca570-04be-41cb-a618-9d72d1b57cbd None None] MFA Rules not processed for user `4751abc9f67947b3ad17ba2b78ce2c12`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:34.026861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d32ca570-04be-41cb-a618-9d72d1b57cbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:34.036974 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1124/2244] 10.222.0.85 () {58 vars in 1018 bytes} [Wed May 25 16:10:34 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:10:34.047577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1621b11-1dbc-4a08-9b1b-418b71f2d267 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:34.059188 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1621b11-1dbc-4a08-9b1b-418b71f2d267 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:34.062872 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391701c-25fa-4a7b-a5c3-e7c51ded2221 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:34.063518 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391701c-25fa-4a7b-a5c3-e7c51ded2221 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:34.063808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391701c-25fa-4a7b-a5c3-e7c51ded2221 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:34.064124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1391701c-25fa-4a7b-a5c3-e7c51ded2221 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:34.075520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391701c-25fa-4a7b-a5c3-e7c51ded2221 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:34.077043 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1391701c-25fa-4a7b-a5c3-e7c51ded2221 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:34.088280 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1121/2245] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:10:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:10:44.019999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=55 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:44.083644 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=64 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:45.535112 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786b5d4-70d9-424e-8021-58445fc07b07 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:45.535627 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786b5d4-70d9-424e-8021-58445fc07b07 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:45.536026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4786b5d4-70d9-424e-8021-58445fc07b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:45.586679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4786b5d4-70d9-424e-8021-58445fc07b07 None None] MFA Rules not processed for user `066de1f11f724eee9df0377beb58d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:45.587911 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4786b5d4-70d9-424e-8021-58445fc07b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:45.604882 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1125/2246] 10.222.0.85 () {58 vars in 1009 bytes} [Wed May 25 16:10:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 71 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:10:45.616851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391701c-25fa-4a7b-a5c3-e7c51ded2221 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:45.638709 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1391701c-25fa-4a7b-a5c3-e7c51ded2221 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:45.644006 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fe4c148-57af-43ba-930c-b24f7789ac41 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77fda27641c84fccb68a5cdaf4e8dc4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:45.644698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4c148-57af-43ba-930c-b24f7789ac41 service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:45.644998 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4c148-57af-43ba-930c-b24f7789ac41 service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:45.645630 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fe4c148-57af-43ba-930c-b24f7789ac41 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:45.654261 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4c148-57af-43ba-930c-b24f7789ac41 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:45.655930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fe4c148-57af-43ba-930c-b24f7789ac41 service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:45.665190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1122/2247] 10.222.0.85 () {60 vars in 1384 bytes} [Wed May 25 16:10:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:10:49.426301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786b5d4-70d9-424e-8021-58445fc07b07 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:49.512561 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4786b5d4-70d9-424e-8021-58445fc07b07 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:49.517274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a645c65-a455-4ca3-a7b5-698ce2ea249f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:49.518725 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a645c65-a455-4ca3-a7b5-698ce2ea249f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:49.518871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a645c65-a455-4ca3-a7b5-698ce2ea249f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:49.519039 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a645c65-a455-4ca3-a7b5-698ce2ea249f admin admin] PATH_INFO: `/v3/users/64110f2238ff42948c0e075afe014b74` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:49.520643 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a645c65-a455-4ca3-a7b5-698ce2ea249f admin admin] RBAC: Authorizing `identity:delete_user(user_id=64110f2238ff42948c0e075afe014b74)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:49.522947 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a645c65-a455-4ca3-a7b5-698ce2ea249f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:49.572992 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5a645c65-a455-4ca3-a7b5-698ce2ea249f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64110f2238ff42948c0e075afe014b74'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:10:49.573654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5a645c65-a455-4ca3-a7b5-698ce2ea249f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64110f2238ff42948c0e075afe014b74'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:10:49.596861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5a645c65-a455-4ca3-a7b5-698ce2ea249f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64110f2238ff42948c0e075afe014b74'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:10:49.611262 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5a645c65-a455-4ca3-a7b5-698ce2ea249f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64110f2238ff42948c0e075afe014b74'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:10:49.611951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5a645c65-a455-4ca3-a7b5-698ce2ea249f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64110f2238ff42948c0e075afe014b74'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:10:49.627502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1126/2248] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:10:49 2022] DELETE /identity/v3/users/64110f2238ff42948c0e075afe014b74 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:10:49.632597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952d113-6fd4-49dd-89e8-c7c9905b2421 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:49.633126 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952d113-6fd4-49dd-89e8-c7c9905b2421 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:49.633209 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c952d113-6fd4-49dd-89e8-c7c9905b2421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:49.655546 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c952d113-6fd4-49dd-89e8-c7c9905b2421 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:49.657803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c952d113-6fd4-49dd-89e8-c7c9905b2421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:49.736156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1123/2249] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:10:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:10:49.745877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a645c65-a455-4ca3-a7b5-698ce2ea249f admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:49.747329 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5a645c65-a455-4ca3-a7b5-698ce2ea249f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:49.795513 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:49.810648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a645c65-a455-4ca3-a7b5-698ce2ea249f admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:49.814551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46d468b-61ab-4e56-b0e0-0c9532fc6d9c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:49.815253 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d468b-61ab-4e56-b0e0-0c9532fc6d9c service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:49.815572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d468b-61ab-4e56-b0e0-0c9532fc6d9c service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:49.815921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46d468b-61ab-4e56-b0e0-0c9532fc6d9c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:49.826737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d468b-61ab-4e56-b0e0-0c9532fc6d9c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:49.828272 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46d468b-61ab-4e56-b0e0-0c9532fc6d9c service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:49.840032 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1127/2250] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:10:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:10:50.135245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952d113-6fd4-49dd-89e8-c7c9905b2421 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:50.137097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c952d113-6fd4-49dd-89e8-c7c9905b2421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:50.151711 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c952d113-6fd4-49dd-89e8-c7c9905b2421 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:50.157541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfad869-6b43-4942-bc32-119f144d3cbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:50.158417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfad869-6b43-4942-bc32-119f144d3cbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:50.158727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfad869-6b43-4942-bc32-119f144d3cbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:50.158987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfad869-6b43-4942-bc32-119f144d3cbd admin admin] PATH_INFO: `/v3/projects/9f70c7a68d8e44b981b62fba5a517703` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:50.160348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfad869-6b43-4942-bc32-119f144d3cbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f70c7a68d8e44b981b62fba5a517703)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:50.162338 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfad869-6b43-4942-bc32-119f144d3cbd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:50.235071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bdfad869-6b43-4942-bc32-119f144d3cbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f70c7a68d8e44b981b62fba5a517703', 'request_id': 'req-bdfad869-6b43-4942-bc32-119f144d3cbd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:10:50.243890 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bdfad869-6b43-4942-bc32-119f144d3cbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f70c7a68d8e44b981b62fba5a517703', 'request_id': 'req-bdfad869-6b43-4942-bc32-119f144d3cbd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:10:50.250773 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bdfad869-6b43-4942-bc32-119f144d3cbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f70c7a68d8e44b981b62fba5a517703', 'request_id': 'req-bdfad869-6b43-4942-bc32-119f144d3cbd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:10:50.262205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bdfad869-6b43-4942-bc32-119f144d3cbd admin admin] The token cache is being invalidate because project 9f70c7a68d8e44b981b62fba5a517703 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:10:50.262528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bdfad869-6b43-4942-bc32-119f144d3cbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:10:50.263197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-bdfad869-6b43-4942-bc32-119f144d3cbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:10:50.264787 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1124/2251] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:10:50 2022] DELETE /identity/v3/projects/9f70c7a68d8e44b981b62fba5a517703 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:10:50.269885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61832c0-f327-4012-98e1-4cc94bca2152 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:50.270531 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61832c0-f327-4012-98e1-4cc94bca2152 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:50.270815 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61832c0-f327-4012-98e1-4cc94bca2152 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:50.291613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e61832c0-f327-4012-98e1-4cc94bca2152 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:50.293553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e61832c0-f327-4012-98e1-4cc94bca2152 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:50.400406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1128/2252] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:10:50.410628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c6fac-ec2c-4056-a4ee-dd9ba8b2f704 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:50.410923 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c6fac-ec2c-4056-a4ee-dd9ba8b2f704 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:50.411205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80c6fac-ec2c-4056-a4ee-dd9ba8b2f704 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:50.430036 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f80c6fac-ec2c-4056-a4ee-dd9ba8b2f704 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:50.431267 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f80c6fac-ec2c-4056-a4ee-dd9ba8b2f704 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:50.439730 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1125/2253] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:10:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:10:50.444956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61832c0-f327-4012-98e1-4cc94bca2152 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:50.460517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61832c0-f327-4012-98e1-4cc94bca2152 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:50.472630 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2d1ff-8357-4e04-9a67-8f4005f1a5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:50.473568 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2d1ff-8357-4e04-9a67-8f4005f1a5fd admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:50.473952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2d1ff-8357-4e04-9a67-8f4005f1a5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:50.474296 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed2d1ff-8357-4e04-9a67-8f4005f1a5fd admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:50.475125 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2d1ff-8357-4e04-9a67-8f4005f1a5fd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:10:50.475513 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2d1ff-8357-4e04-9a67-8f4005f1a5fd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:50.477475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed2d1ff-8357-4e04-9a67-8f4005f1a5fd admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:50.498826 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1129/2254] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:10:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:10:50.500022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c6fac-ec2c-4056-a4ee-dd9ba8b2f704 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:50.510491 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80c6fac-ec2c-4056-a4ee-dd9ba8b2f704 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:50.515433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbb63a-4ec1-43c5-a3f3-4a076f7f491c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:50.516232 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbb63a-4ec1-43c5-a3f3-4a076f7f491c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:50.516525 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbb63a-4ec1-43c5-a3f3-4a076f7f491c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:50.516808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fbb63a-4ec1-43c5-a3f3-4a076f7f491c admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:50.517531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbb63a-4ec1-43c5-a3f3-4a076f7f491c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:50.519529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fbb63a-4ec1-43c5-a3f3-4a076f7f491c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:50.548886 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1126/2255] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:10:50 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:10:50.553168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2d1ff-8357-4e04-9a67-8f4005f1a5fd admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:50.564145 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed2d1ff-8357-4e04-9a67-8f4005f1a5fd admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:50.622319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa340e2b-9e06-41e5-a2ab-f170bfca7ce9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:50.623006 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa340e2b-9e06-41e5-a2ab-f170bfca7ce9 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:50.623291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa340e2b-9e06-41e5-a2ab-f170bfca7ce9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:50.623563 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa340e2b-9e06-41e5-a2ab-f170bfca7ce9 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:50.624310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa340e2b-9e06-41e5-a2ab-f170bfca7ce9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:50.626484 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa340e2b-9e06-41e5-a2ab-f170bfca7ce9 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:50.651928 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1130/2256] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:10:50 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:10:50.654900 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbb63a-4ec1-43c5-a3f3-4a076f7f491c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:50.667303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fbb63a-4ec1-43c5-a3f3-4a076f7f491c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:50.673271 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5e4b6-3b44-4a0f-ba4d-488006142dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:50.674197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5e4b6-3b44-4a0f-ba4d-488006142dbb admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:50.674497 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5e4b6-3b44-4a0f-ba4d-488006142dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:50.674763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c5e4b6-3b44-4a0f-ba4d-488006142dbb admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:50.675568 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5e4b6-3b44-4a0f-ba4d-488006142dbb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:50.677435 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c5e4b6-3b44-4a0f-ba4d-488006142dbb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:50.689066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1127/2257] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:10:50.693364 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa340e2b-9e06-41e5-a2ab-f170bfca7ce9 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:50.704223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa340e2b-9e06-41e5-a2ab-f170bfca7ce9 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:50.709357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b00f54af-eb8f-4daf-b29d-9d0e9cca2911 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:50.710060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f54af-eb8f-4daf-b29d-9d0e9cca2911 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:50.710360 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f54af-eb8f-4daf-b29d-9d0e9cca2911 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:50.710659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00f54af-eb8f-4daf-b29d-9d0e9cca2911 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:50.711417 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f54af-eb8f-4daf-b29d-9d0e9cca2911 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:50.713122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b00f54af-eb8f-4daf-b29d-9d0e9cca2911 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:50.728143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1131/2258] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:10:50 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:10:50.742832 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5e4b6-3b44-4a0f-ba4d-488006142dbb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:50.753447 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c5e4b6-3b44-4a0f-ba4d-488006142dbb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:50.758204 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b3914-d751-4c1f-9968-28c4da5cd30b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:50.759107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b3914-d751-4c1f-9968-28c4da5cd30b admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:50.759394 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b3914-d751-4c1f-9968-28c4da5cd30b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:50.759675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b3914-d751-4c1f-9968-28c4da5cd30b admin admin] PATH_INFO: `/v3/projects/2e01367e6f0a4bb9abf67383702ae6c2/users/86a27ac3f2ba45469d8b9a871e4df159/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:50.776495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b3914-d751-4c1f-9968-28c4da5cd30b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2e01367e6f0a4bb9abf67383702ae6c2, user_id=86a27ac3f2ba45469d8b9a871e4df159, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:50.779237 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082b3914-d751-4c1f-9968-28c4da5cd30b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:50.789177 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:50.795279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1128/2259] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:10:50 2022] PUT /identity/v3/projects/2e01367e6f0a4bb9abf67383702ae6c2/users/86a27ac3f2ba45469d8b9a871e4df159/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:10:50.805863 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9811c82-719d-4f0a-b726-06a926627836 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:50.806163 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9811c82-719d-4f0a-b726-06a926627836 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:50.806446 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9811c82-719d-4f0a-b726-06a926627836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:50.840561 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f9811c82-719d-4f0a-b726-06a926627836 None None] MFA Rules not processed for user `86a27ac3f2ba45469d8b9a871e4df159`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:50.841853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f9811c82-719d-4f0a-b726-06a926627836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:50.886687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:10:50.915733 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1132/2260] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:10:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:10:50.926535 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b3914-d751-4c1f-9968-28c4da5cd30b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:50.928336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-082b3914-d751-4c1f-9968-28c4da5cd30b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:51.006566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082b3914-d751-4c1f-9968-28c4da5cd30b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:51.010440 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd46db-9160-4252-a4e1-fee84e9f9b0e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:51.011097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd46db-9160-4252-a4e1-fee84e9f9b0e service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:51.011393 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd46db-9160-4252-a4e1-fee84e9f9b0e service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:51.011675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fd46db-9160-4252-a4e1-fee84e9f9b0e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:51.020938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd46db-9160-4252-a4e1-fee84e9f9b0e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:51.022588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fd46db-9160-4252-a4e1-fee84e9f9b0e service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:51.039368 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1129/2261] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:10:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:10:52.432835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913986cf-cd3c-46dc-bd2a-f1db234dca33 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:52.433373 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913986cf-cd3c-46dc-bd2a-f1db234dca33 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:52.433477 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-913986cf-cd3c-46dc-bd2a-f1db234dca33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:52.451594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-913986cf-cd3c-46dc-bd2a-f1db234dca33 None None] MFA Rules not processed for user `86a27ac3f2ba45469d8b9a871e4df159`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:10:52.452785 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-913986cf-cd3c-46dc-bd2a-f1db234dca33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:52.459775 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1133/2262] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:10:52 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:10:58.158248 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd46db-9160-4252-a4e1-fee84e9f9b0e service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:10:58.159920 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f4fd46db-9160-4252-a4e1-fee84e9f9b0e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:58.272609 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fd46db-9160-4252-a4e1-fee84e9f9b0e service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:10:58.278414 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8607e5-55e9-40b5-b372-190a3904e18f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b738f1b8c64590955a640c461b4ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:10:58.279071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8607e5-55e9-40b5-b372-190a3904e18f service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:10:58.279374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8607e5-55e9-40b5-b372-190a3904e18f service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:10:58.279659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c8607e5-55e9-40b5-b372-190a3904e18f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:10:58.281446 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4c8607e5-55e9-40b5-b372-190a3904e18f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:10:58.293266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8607e5-55e9-40b5-b372-190a3904e18f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:10:58.294884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c8607e5-55e9-40b5-b372-190a3904e18f service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:10:58.305840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1130/2263] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:10:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:11:02.446270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=13 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:08.302193 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=77 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:13.764854 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178de44-2db3-45b6-a40f-a846e87b8ecc None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:13.765346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178de44-2db3-45b6-a40f-a846e87b8ecc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:13.765436 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f178de44-2db3-45b6-a40f-a846e87b8ecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:13.786070 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f178de44-2db3-45b6-a40f-a846e87b8ecc None None] MFA Rules not processed for user `066de1f11f724eee9df0377beb58d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:13.787403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f178de44-2db3-45b6-a40f-a846e87b8ecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:13.844367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1134/2264] 10.222.0.85 () {58 vars in 967 bytes} [Wed May 25 16:11:13 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:13.857643 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8607e5-55e9-40b5-b372-190a3904e18f service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:11:13.859963 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4c8607e5-55e9-40b5-b372-190a3904e18f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:13.960397 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c8607e5-55e9-40b5-b372-190a3904e18f service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:11:13.966163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e479e3ab-3943-4104-add7-dc2819d90d07 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77fda27641c84fccb68a5cdaf4e8dc4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:11:13.966815 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479e3ab-3943-4104-add7-dc2819d90d07 service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:13.967113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479e3ab-3943-4104-add7-dc2819d90d07 service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:13.967388 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e479e3ab-3943-4104-add7-dc2819d90d07 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:13.975948 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479e3ab-3943-4104-add7-dc2819d90d07 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:11:13.977565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e479e3ab-3943-4104-add7-dc2819d90d07 service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:11:13.985912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1131/2265] 10.222.0.85 () {60 vars in 1384 bytes} [Wed May 25 16:11:13 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:11:18.698630 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e483eda-3294-4ac8-a9e2-4435342bd042 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:18.699229 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e483eda-3294-4ac8-a9e2-4435342bd042 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:18.699229 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e483eda-3294-4ac8-a9e2-4435342bd042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:18.721547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8e483eda-3294-4ac8-a9e2-4435342bd042 None None] MFA Rules not processed for user `2b909d4c7d9b45b29f995268f8fac987`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:18.722800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8e483eda-3294-4ac8-a9e2-4435342bd042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:18.783903 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1135/2266] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:11:18 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:23.982762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=37 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:24.733601 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd1997-f980-4161-ac8e-0b3572161cd0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:24.733901 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd1997-f980-4161-ac8e-0b3572161cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:24.734192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27fd1997-f980-4161-ac8e-0b3572161cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:24.752730 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-27fd1997-f980-4161-ac8e-0b3572161cd0 None None] MFA Rules not processed for user `066de1f11f724eee9df0377beb58d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:24.753942 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-27fd1997-f980-4161-ac8e-0b3572161cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:24.761417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1132/2267] 10.222.0.85 () {58 vars in 1009 bytes} [Wed May 25 16:11:24 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:24.783063 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e483eda-3294-4ac8-a9e2-4435342bd042 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:11:24.795535 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e483eda-3294-4ac8-a9e2-4435342bd042 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:11:24.801317 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9075de-1be6-4ba3-9689-e6345de07ed5 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77fda27641c84fccb68a5cdaf4e8dc4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:11:24.801901 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9075de-1be6-4ba3-9689-e6345de07ed5 service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:24.802197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9075de-1be6-4ba3-9689-e6345de07ed5 service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:24.802469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec9075de-1be6-4ba3-9689-e6345de07ed5 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:24.811020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9075de-1be6-4ba3-9689-e6345de07ed5 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:11:24.812684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec9075de-1be6-4ba3-9689-e6345de07ed5 service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:11:24.821163 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1136/2268] 10.222.0.85 () {60 vars in 1384 bytes} [Wed May 25 16:11:24 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:11:25.366116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e6064-3c5b-414e-be7d-02999eb683f6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:25.366585 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e6064-3c5b-414e-be7d-02999eb683f6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:25.366841 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9e6064-3c5b-414e-be7d-02999eb683f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:25.384986 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ce9e6064-3c5b-414e-be7d-02999eb683f6 None None] MFA Rules not processed for user `066de1f11f724eee9df0377beb58d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:25.386222 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ce9e6064-3c5b-414e-be7d-02999eb683f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:25.394218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1133/2269] 10.222.0.85 () {58 vars in 967 bytes} [Wed May 25 16:11:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:25.406338 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9075de-1be6-4ba3-9689-e6345de07ed5 service swift] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:11:25.416830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec9075de-1be6-4ba3-9689-e6345de07ed5 service swift] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:11:25.422351 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18dd92-0dba-4c95-993a-8e7b1bc6e353 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77fda27641c84fccb68a5cdaf4e8dc4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:11:25.422989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18dd92-0dba-4c95-993a-8e7b1bc6e353 service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:25.423275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18dd92-0dba-4c95-993a-8e7b1bc6e353 service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:25.423552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a18dd92-0dba-4c95-993a-8e7b1bc6e353 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:25.432435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18dd92-0dba-4c95-993a-8e7b1bc6e353 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:11:25.434164 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a18dd92-0dba-4c95-993a-8e7b1bc6e353 service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:11:25.442377 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1137/2270] 10.222.0.85 () {60 vars in 1384 bytes} [Wed May 25 16:11:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:11:25.642144 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7e5d2-cf1c-4b9b-aad5-993f8e125835 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:25.642598 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7e5d2-cf1c-4b9b-aad5-993f8e125835 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:25.643281 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a7e5d2-cf1c-4b9b-aad5-993f8e125835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:25.661728 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d4a7e5d2-cf1c-4b9b-aad5-993f8e125835 None None] MFA Rules not processed for user `066de1f11f724eee9df0377beb58d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:25.662941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d4a7e5d2-cf1c-4b9b-aad5-993f8e125835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:25.670427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1134/2271] 10.222.0.85 () {58 vars in 1009 bytes} [Wed May 25 16:11:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:25.682105 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18dd92-0dba-4c95-993a-8e7b1bc6e353 service swift] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:11:25.693294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a18dd92-0dba-4c95-993a-8e7b1bc6e353 service swift] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:11:25.699669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017052a-df22-4d34-80be-da5c43deab66 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77fda27641c84fccb68a5cdaf4e8dc4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:11:25.700427 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017052a-df22-4d34-80be-da5c43deab66 service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:25.700712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017052a-df22-4d34-80be-da5c43deab66 service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:25.700994 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9017052a-df22-4d34-80be-da5c43deab66 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:25.710114 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017052a-df22-4d34-80be-da5c43deab66 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:11:25.711754 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9017052a-df22-4d34-80be-da5c43deab66 service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:11:25.721462 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1138/2272] 10.222.0.85 () {60 vars in 1384 bytes} [Wed May 25 16:11:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:11:26.063970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbab001-a0d5-458d-90bc-e9a1f028608e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:26.064271 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbab001-a0d5-458d-90bc-e9a1f028608e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:26.064557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cbab001-a0d5-458d-90bc-e9a1f028608e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:26.085439 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3cbab001-a0d5-458d-90bc-e9a1f028608e None None] MFA Rules not processed for user `066de1f11f724eee9df0377beb58d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:26.086905 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3cbab001-a0d5-458d-90bc-e9a1f028608e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:26.095136 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1135/2273] 10.222.0.85 () {58 vars in 967 bytes} [Wed May 25 16:11:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:26.107779 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017052a-df22-4d34-80be-da5c43deab66 service swift] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:11:26.118577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9017052a-df22-4d34-80be-da5c43deab66 service swift] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:11:26.123876 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824f6e7d-9346-4c39-883b-6ade901aa9bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77fda27641c84fccb68a5cdaf4e8dc4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:11:26.124707 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f6e7d-9346-4c39-883b-6ade901aa9bd service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:26.125029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f6e7d-9346-4c39-883b-6ade901aa9bd service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:26.125306 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824f6e7d-9346-4c39-883b-6ade901aa9bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:26.133634 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f6e7d-9346-4c39-883b-6ade901aa9bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:11:26.135219 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824f6e7d-9346-4c39-883b-6ade901aa9bd service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:11:26.143654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1139/2274] 10.222.0.85 () {60 vars in 1384 bytes} [Wed May 25 16:11:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:11:28.238746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbab001-a0d5-458d-90bc-e9a1f028608e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:11:28.308116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cbab001-a0d5-458d-90bc-e9a1f028608e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:11:28.313646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d509e-5e23-4c32-8342-365f460239aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:11:28.314554 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d509e-5e23-4c32-8342-365f460239aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:28.314850 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d509e-5e23-4c32-8342-365f460239aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:28.315109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-671d509e-5e23-4c32-8342-365f460239aa admin admin] PATH_INFO: `/v3/users/86a27ac3f2ba45469d8b9a871e4df159` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:28.316446 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d509e-5e23-4c32-8342-365f460239aa admin admin] RBAC: Authorizing `identity:delete_user(user_id=86a27ac3f2ba45469d8b9a871e4df159)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:11:28.318382 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-671d509e-5e23-4c32-8342-365f460239aa admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:11:28.375381 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-671d509e-5e23-4c32-8342-365f460239aa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '86a27ac3f2ba45469d8b9a871e4df159'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:11:28.387770 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-671d509e-5e23-4c32-8342-365f460239aa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '86a27ac3f2ba45469d8b9a871e4df159'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:11:28.397373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-671d509e-5e23-4c32-8342-365f460239aa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '86a27ac3f2ba45469d8b9a871e4df159'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:11:28.398128 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-671d509e-5e23-4c32-8342-365f460239aa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '86a27ac3f2ba45469d8b9a871e4df159'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:11:28.405932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-671d509e-5e23-4c32-8342-365f460239aa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '86a27ac3f2ba45469d8b9a871e4df159'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:11:28.411981 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1136/2275] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:11:28 2022] DELETE /identity/v3/users/86a27ac3f2ba45469d8b9a871e4df159 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:11:28.418015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c8f62-5804-4a4f-bd57-a43b859ba403 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:28.418543 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c8f62-5804-4a4f-bd57-a43b859ba403 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:28.418657 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c8f62-5804-4a4f-bd57-a43b859ba403 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:28.441794 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-948c8f62-5804-4a4f-bd57-a43b859ba403 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:28.442991 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-948c8f62-5804-4a4f-bd57-a43b859ba403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:28.504886 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:28.517983 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1140/2276] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:11:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:28.530086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d509e-5e23-4c32-8342-365f460239aa admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:11:28.531721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-671d509e-5e23-4c32-8342-365f460239aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:28.603626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-671d509e-5e23-4c32-8342-365f460239aa admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:11:28.607766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0cb2a3-7b98-4f83-8c60-33482e8a2c06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:11:28.608465 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cb2a3-7b98-4f83-8c60-33482e8a2c06 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:28.608760 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cb2a3-7b98-4f83-8c60-33482e8a2c06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:28.609046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0cb2a3-7b98-4f83-8c60-33482e8a2c06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:28.616930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:28.619612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cb2a3-7b98-4f83-8c60-33482e8a2c06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:11:28.621225 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0cb2a3-7b98-4f83-8c60-33482e8a2c06 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:11:28.640443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1137/2277] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:11:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:11:28.839412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c8f62-5804-4a4f-bd57-a43b859ba403 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:11:28.841058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-948c8f62-5804-4a4f-bd57-a43b859ba403 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:28.853945 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c8f62-5804-4a4f-bd57-a43b859ba403 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:11:28.859545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd980f9-a53d-4c87-b85e-6c4b4d057887 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:11:28.860589 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd980f9-a53d-4c87-b85e-6c4b4d057887 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:28.860737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd980f9-a53d-4c87-b85e-6c4b4d057887 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:28.861018 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd980f9-a53d-4c87-b85e-6c4b4d057887 admin admin] PATH_INFO: `/v3/projects/2e01367e6f0a4bb9abf67383702ae6c2` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:28.862359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd980f9-a53d-4c87-b85e-6c4b4d057887 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2e01367e6f0a4bb9abf67383702ae6c2)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:11:28.864319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd980f9-a53d-4c87-b85e-6c4b4d057887 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:11:28.915376 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bd980f9-a53d-4c87-b85e-6c4b4d057887 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2e01367e6f0a4bb9abf67383702ae6c2', 'request_id': 'req-0bd980f9-a53d-4c87-b85e-6c4b4d057887'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:11:28.920858 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bd980f9-a53d-4c87-b85e-6c4b4d057887 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2e01367e6f0a4bb9abf67383702ae6c2', 'request_id': 'req-0bd980f9-a53d-4c87-b85e-6c4b4d057887'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:11:28.929866 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bd980f9-a53d-4c87-b85e-6c4b4d057887 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2e01367e6f0a4bb9abf67383702ae6c2', 'request_id': 'req-0bd980f9-a53d-4c87-b85e-6c4b4d057887'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:11:28.941586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bd980f9-a53d-4c87-b85e-6c4b4d057887 admin admin] The token cache is being invalidate because project 2e01367e6f0a4bb9abf67383702ae6c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:11:28.941912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bd980f9-a53d-4c87-b85e-6c4b4d057887 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:11:28.942647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-0bd980f9-a53d-4c87-b85e-6c4b4d057887 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:11:28.944445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1141/2278] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:11:28 2022] DELETE /identity/v3/projects/2e01367e6f0a4bb9abf67383702ae6c2 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:11:38.634514 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:38.938625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=28,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:11:55.442305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b1d87-1177-48a3-9378-70eb33f1673d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:55.442836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b1d87-1177-48a3-9378-70eb33f1673d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:55.442934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488b1d87-1177-48a3-9378-70eb33f1673d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:55.460996 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-488b1d87-1177-48a3-9378-70eb33f1673d None None] MFA Rules not processed for user `708fdc537e1747d7bd5917ed7af9c368`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:55.462200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-488b1d87-1177-48a3-9378-70eb33f1673d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:55.518719 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1138/2279] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:11:55 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:11:56.332196 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6364d-7992-4d92-b6a6-a3ca33413d20 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:11:56.333297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6364d-7992-4d92-b6a6-a3ca33413d20 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:11:56.333747 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f6364d-7992-4d92-b6a6-a3ca33413d20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:11:56.367548 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-a8f6364d-7992-4d92-b6a6-a3ca33413d20 None None] MFA Rules not processed for user `d0427d9cce40423185a40df61c3617a6`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:11:56.368899 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a8f6364d-7992-4d92-b6a6-a3ca33413d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:11:56.457716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1142/2280] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:11:56 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:12:05.511595 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=25 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:12:06.450324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=25 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:12:21.096598 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fecd95-6ced-4fff-960d-8c761ef2a201 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:21.097168 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fecd95-6ced-4fff-960d-8c761ef2a201 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:21.097275 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9fecd95-6ced-4fff-960d-8c761ef2a201 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:21.115366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b9fecd95-6ced-4fff-960d-8c761ef2a201 None None] MFA Rules not processed for user `be5c51fdd2fe4159a83a560dc074da6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:12:21.116638 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b9fecd95-6ced-4fff-960d-8c761ef2a201 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:21.172777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1139/2281] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:12:21 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:12:26.198555 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6364d-7992-4d92-b6a6-a3ca33413d20 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:26.200271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a8f6364d-7992-4d92-b6a6-a3ca33413d20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:26.270139 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f6364d-7992-4d92-b6a6-a3ca33413d20 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:26.275125 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d2266-e142-4303-b069-bd541e10c7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:26.276054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d2266-e142-4303-b069-bd541e10c7f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:26.276341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d2266-e142-4303-b069-bd541e10c7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:26.276621 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-534d2266-e142-4303-b069-bd541e10c7f0 admin admin] PATH_INFO: `/v3/users/2b909d4c7d9b45b29f995268f8fac987` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:26.277853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d2266-e142-4303-b069-bd541e10c7f0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b909d4c7d9b45b29f995268f8fac987)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:26.279844 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-534d2266-e142-4303-b069-bd541e10c7f0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:26.332526 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-534d2266-e142-4303-b069-bd541e10c7f0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b909d4c7d9b45b29f995268f8fac987'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:12:26.333268 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-534d2266-e142-4303-b069-bd541e10c7f0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b909d4c7d9b45b29f995268f8fac987'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:12:26.346566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-534d2266-e142-4303-b069-bd541e10c7f0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b909d4c7d9b45b29f995268f8fac987'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:12:26.355648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-534d2266-e142-4303-b069-bd541e10c7f0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b909d4c7d9b45b29f995268f8fac987'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:12:26.356288 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-534d2266-e142-4303-b069-bd541e10c7f0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b909d4c7d9b45b29f995268f8fac987'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:12:26.368732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1143/2282] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:12:26 2022] DELETE /identity/v3/users/2b909d4c7d9b45b29f995268f8fac987 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:12:26.546055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fecd95-6ced-4fff-960d-8c761ef2a201 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:26.561705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9fecd95-6ced-4fff-960d-8c761ef2a201 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:26.633017 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818226d-5607-446b-8167-e152a19d9f8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:26.633816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818226d-5607-446b-8167-e152a19d9f8b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:26.634103 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818226d-5607-446b-8167-e152a19d9f8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:26.634390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6818226d-5607-446b-8167-e152a19d9f8b admin admin] PATH_INFO: `/v3/projects/b32433ec45854be380fcd9a9671a4ce5` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:26.635602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818226d-5607-446b-8167-e152a19d9f8b admin admin] RBAC: Authorizing `identity:delete_project(project_id=b32433ec45854be380fcd9a9671a4ce5)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:26.637584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6818226d-5607-446b-8167-e152a19d9f8b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:26.688800 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6818226d-5607-446b-8167-e152a19d9f8b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b32433ec45854be380fcd9a9671a4ce5', 'request_id': 'req-6818226d-5607-446b-8167-e152a19d9f8b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:12:26.696004 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6818226d-5607-446b-8167-e152a19d9f8b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b32433ec45854be380fcd9a9671a4ce5', 'request_id': 'req-6818226d-5607-446b-8167-e152a19d9f8b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:12:26.702692 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6818226d-5607-446b-8167-e152a19d9f8b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b32433ec45854be380fcd9a9671a4ce5', 'request_id': 'req-6818226d-5607-446b-8167-e152a19d9f8b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:12:26.713735 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6818226d-5607-446b-8167-e152a19d9f8b admin admin] The token cache is being invalidate because project b32433ec45854be380fcd9a9671a4ce5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:12:26.714024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6818226d-5607-446b-8167-e152a19d9f8b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:12:26.714712 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6818226d-5607-446b-8167-e152a19d9f8b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:12:26.716331 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1140/2283] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:12:26 2022] DELETE /identity/v3/projects/b32433ec45854be380fcd9a9671a4ce5 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:12:36.365383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:12:36.710962 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=76,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:12:50.524092 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d2266-e142-4303-b069-bd541e10c7f0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:50.525721 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-534d2266-e142-4303-b069-bd541e10c7f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:50.593737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-534d2266-e142-4303-b069-bd541e10c7f0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:50.654096 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928e2fc-e1b6-40b7-ae55-44330de8d89b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:50.654757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928e2fc-e1b6-40b7-ae55-44330de8d89b service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:50.655052 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928e2fc-e1b6-40b7-ae55-44330de8d89b service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:50.655337 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a928e2fc-e1b6-40b7-ae55-44330de8d89b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:50.657329 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a928e2fc-e1b6-40b7-ae55-44330de8d89b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:50.730074 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928e2fc-e1b6-40b7-ae55-44330de8d89b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:50.731534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a928e2fc-e1b6-40b7-ae55-44330de8d89b service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:50.742681 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1144/2284] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:12:50 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:51.937916 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818226d-5607-446b-8167-e152a19d9f8b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:51.939474 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6818226d-5607-446b-8167-e152a19d9f8b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:51.952162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6818226d-5607-446b-8167-e152a19d9f8b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:51.956943 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09bbb820-6600-4792-a79d-f2b25cfc63ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:51.957745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbb820-6600-4792-a79d-f2b25cfc63ec service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:51.958034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbb820-6600-4792-a79d-f2b25cfc63ec service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:51.958323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09bbb820-6600-4792-a79d-f2b25cfc63ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:51.960244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-09bbb820-6600-4792-a79d-f2b25cfc63ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:12:51.974189 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bbb820-6600-4792-a79d-f2b25cfc63ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:12:51.975626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09bbb820-6600-4792-a79d-f2b25cfc63ec service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:12:51.986775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1141/2285] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:12:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:12:59.916566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928e2fc-e1b6-40b7-ae55-44330de8d89b service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:12:59.930103 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a928e2fc-e1b6-40b7-ae55-44330de8d89b service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:12:59.935557 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8806486-9230-48e6-bbbd-e06f40e6846a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:12:59.936224 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8806486-9230-48e6-bbbd-e06f40e6846a service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:12:59.936521 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8806486-9230-48e6-bbbd-e06f40e6846a service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:12:59.936842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8806486-9230-48e6-bbbd-e06f40e6846a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:12:59.939603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d8806486-9230-48e6-bbbd-e06f40e6846a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:00.011420 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:00.015201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8806486-9230-48e6-bbbd-e06f40e6846a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:00.016989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8806486-9230-48e6-bbbd-e06f40e6846a service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:00.028336 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1145/2286] 10.222.0.85 () {60 vars in 1396 bytes} [Wed May 25 16:12:59 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3258 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:01.981879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:03.519758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce7223-872f-4dd2-87f3-1d0e27028055 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:03.520362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce7223-872f-4dd2-87f3-1d0e27028055 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:03.520362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cce7223-872f-4dd2-87f3-1d0e27028055 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:03.538639 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3cce7223-872f-4dd2-87f3-1d0e27028055 None None] MFA Rules not processed for user `a0289402c6f245839a86b7b66714a8f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:03.539949 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3cce7223-872f-4dd2-87f3-1d0e27028055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:03.595128 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1142/2287] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:03 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:03.899662 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8806486-9230-48e6-bbbd-e06f40e6846a service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:03.908547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8806486-9230-48e6-bbbd-e06f40e6846a service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:03.912495 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c3a68-ba1f-447a-804e-6e0330dda4fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:03.913147 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c3a68-ba1f-447a-804e-6e0330dda4fc service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:03.913439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c3a68-ba1f-447a-804e-6e0330dda4fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:03.913723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2c3a68-ba1f-447a-804e-6e0330dda4fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:03.915802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7e2c3a68-ba1f-447a-804e-6e0330dda4fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:03.926084 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c3a68-ba1f-447a-804e-6e0330dda4fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:03.927523 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e2c3a68-ba1f-447a-804e-6e0330dda4fc service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:03.935551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1146/2288] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:03 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:04.260938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce7223-872f-4dd2-87f3-1d0e27028055 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:04.262673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3cce7223-872f-4dd2-87f3-1d0e27028055 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:04.334681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cce7223-872f-4dd2-87f3-1d0e27028055 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:04.340630 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12856ad6-0a00-4235-ad35-1ce7b07a855b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b738f1b8c64590955a640c461b4ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:04.341290 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12856ad6-0a00-4235-ad35-1ce7b07a855b service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:04.341584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12856ad6-0a00-4235-ad35-1ce7b07a855b service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:04.341868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12856ad6-0a00-4235-ad35-1ce7b07a855b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:04.344116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-12856ad6-0a00-4235-ad35-1ce7b07a855b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:04.355746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12856ad6-0a00-4235-ad35-1ce7b07a855b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:04.357837 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12856ad6-0a00-4235-ad35-1ce7b07a855b service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:04.370720 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1143/2289] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:04 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:06.572326 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c3a68-ba1f-447a-804e-6e0330dda4fc service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:06.573912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7e2c3a68-ba1f-447a-804e-6e0330dda4fc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:06.586541 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e2c3a68-ba1f-447a-804e-6e0330dda4fc service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:06.591367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00430fd-686e-4558-a9e0-36fcd1788742 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:06.592046 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00430fd-686e-4558-a9e0-36fcd1788742 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:06.592351 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00430fd-686e-4558-a9e0-36fcd1788742 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:06.592643 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00430fd-686e-4558-a9e0-36fcd1788742 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:06.594441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e00430fd-686e-4558-a9e0-36fcd1788742 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:06.663951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00430fd-686e-4558-a9e0-36fcd1788742 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:06.665408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00430fd-686e-4558-a9e0-36fcd1788742 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:06.675261 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1147/2290] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:07.877673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12856ad6-0a00-4235-ad35-1ce7b07a855b service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:07.879196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-12856ad6-0a00-4235-ad35-1ce7b07a855b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:07.946141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12856ad6-0a00-4235-ad35-1ce7b07a855b service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:07.951043 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a15de-b4cf-468c-a21e-1308a5964988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:07.952068 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a15de-b4cf-468c-a21e-1308a5964988 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:07.952926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a15de-b4cf-468c-a21e-1308a5964988 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:07.953225 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4a15de-b4cf-468c-a21e-1308a5964988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:07.955959 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-db4a15de-b4cf-468c-a21e-1308a5964988 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:08.010659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:08.023772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a15de-b4cf-468c-a21e-1308a5964988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:08.025288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4a15de-b4cf-468c-a21e-1308a5964988 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:08.034520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1144/2291] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:07 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:08.986609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00430fd-686e-4558-a9e0-36fcd1788742 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:08.988340 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e00430fd-686e-4558-a9e0-36fcd1788742 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:09.005333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00430fd-686e-4558-a9e0-36fcd1788742 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:09.012679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cd3bd-5998-4753-a3e5-c8c44f46d2a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:09.013646 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cd3bd-5998-4753-a3e5-c8c44f46d2a4 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:09.015399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cd3bd-5998-4753-a3e5-c8c44f46d2a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:09.015671 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c7cd3bd-5998-4753-a3e5-c8c44f46d2a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:09.018603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2c7cd3bd-5998-4753-a3e5-c8c44f46d2a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:09.030314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cd3bd-5998-4753-a3e5-c8c44f46d2a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:09.032104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c7cd3bd-5998-4753-a3e5-c8c44f46d2a4 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:09.044074 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1148/2292] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:08 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:10.087918 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a15de-b4cf-468c-a21e-1308a5964988 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:10.089980 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-db4a15de-b4cf-468c-a21e-1308a5964988 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:10.104253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4a15de-b4cf-468c-a21e-1308a5964988 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:10.108777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6276adab-b4da-478f-83ad-050d01559ff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:10.109661 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276adab-b4da-478f-83ad-050d01559ff4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:10.109951 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276adab-b4da-478f-83ad-050d01559ff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:10.110300 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6276adab-b4da-478f-83ad-050d01559ff4 admin admin] PATH_INFO: `/v3/users/d0427d9cce40423185a40df61c3617a6` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:10.111502 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276adab-b4da-478f-83ad-050d01559ff4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d0427d9cce40423185a40df61c3617a6)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:10.113498 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6276adab-b4da-478f-83ad-050d01559ff4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:10.172840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6276adab-b4da-478f-83ad-050d01559ff4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd0427d9cce40423185a40df61c3617a6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:10.188052 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6276adab-b4da-478f-83ad-050d01559ff4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd0427d9cce40423185a40df61c3617a6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:10.205594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6276adab-b4da-478f-83ad-050d01559ff4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd0427d9cce40423185a40df61c3617a6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:10.206253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6276adab-b4da-478f-83ad-050d01559ff4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd0427d9cce40423185a40df61c3617a6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:10.213997 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6276adab-b4da-478f-83ad-050d01559ff4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd0427d9cce40423185a40df61c3617a6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:10.220167 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1145/2293] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:13:10 2022] DELETE /identity/v3/users/d0427d9cce40423185a40df61c3617a6 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:10.451422 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cd3bd-5998-4753-a3e5-c8c44f46d2a4 service nova] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:10.462430 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c7cd3bd-5998-4753-a3e5-c8c44f46d2a4 service nova] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:10.524019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9133fde1-cdc5-4251-967a-e20a142810ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:10.525203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133fde1-cdc5-4251-967a-e20a142810ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:10.525566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133fde1-cdc5-4251-967a-e20a142810ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:10.526534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9133fde1-cdc5-4251-967a-e20a142810ba admin admin] PATH_INFO: `/v3/projects/f7733d8b5059433eb18dc6b174eb9533` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:10.527726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133fde1-cdc5-4251-967a-e20a142810ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=f7733d8b5059433eb18dc6b174eb9533)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:10.529690 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9133fde1-cdc5-4251-967a-e20a142810ba admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:10.576102 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9133fde1-cdc5-4251-967a-e20a142810ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f7733d8b5059433eb18dc6b174eb9533', 'request_id': 'req-9133fde1-cdc5-4251-967a-e20a142810ba'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:10.582312 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9133fde1-cdc5-4251-967a-e20a142810ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f7733d8b5059433eb18dc6b174eb9533', 'request_id': 'req-9133fde1-cdc5-4251-967a-e20a142810ba'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:10.591115 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9133fde1-cdc5-4251-967a-e20a142810ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f7733d8b5059433eb18dc6b174eb9533', 'request_id': 'req-9133fde1-cdc5-4251-967a-e20a142810ba'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:10.592287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:10.604879 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9133fde1-cdc5-4251-967a-e20a142810ba admin admin] The token cache is being invalidate because project f7733d8b5059433eb18dc6b174eb9533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:13:10.605173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9133fde1-cdc5-4251-967a-e20a142810ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:10.605821 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9133fde1-cdc5-4251-967a-e20a142810ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:10.607416 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1149/2294] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:13:10 2022] DELETE /identity/v3/projects/f7733d8b5059433eb18dc6b174eb9533 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:13:10.612461 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8008c-1b39-4395-9eeb-f1a3a55ba1e8 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:10.612754 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8008c-1b39-4395-9eeb-f1a3a55ba1e8 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:10.613043 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66e8008c-1b39-4395-9eeb-f1a3a55ba1e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:10.632792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-66e8008c-1b39-4395-9eeb-f1a3a55ba1e8 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:10.634049 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-66e8008c-1b39-4395-9eeb-f1a3a55ba1e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:10.699797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1146/2295] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:10.708109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc098494-ecc1-41c2-80f1-a4bd56174890 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:10.708445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc098494-ecc1-41c2-80f1-a4bd56174890 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:10.708730 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc098494-ecc1-41c2-80f1-a4bd56174890 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:10.727919 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-bc098494-ecc1-41c2-80f1-a4bd56174890 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:10.729061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-bc098494-ecc1-41c2-80f1-a4bd56174890 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:10.738200 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1150/2296] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:13:10.742710 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8008c-1b39-4395-9eeb-f1a3a55ba1e8 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:10.753422 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66e8008c-1b39-4395-9eeb-f1a3a55ba1e8 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:10.758399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ff909-6389-417a-afb6-e14322d671ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:10.759086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ff909-6389-417a-afb6-e14322d671ec admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:10.759380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ff909-6389-417a-afb6-e14322d671ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:10.759672 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b92ff909-6389-417a-afb6-e14322d671ec admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:10.760490 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ff909-6389-417a-afb6-e14322d671ec admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:13:10.760827 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ff909-6389-417a-afb6-e14322d671ec admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:10.762868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b92ff909-6389-417a-afb6-e14322d671ec admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:10.775067 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1147/2297] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:13:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:13:10.779670 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc098494-ecc1-41c2-80f1-a4bd56174890 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:10.789272 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc098494-ecc1-41c2-80f1-a4bd56174890 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:10.794382 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff8b57-f70e-4478-894a-5f6d9ea7c1eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:10.795027 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff8b57-f70e-4478-894a-5f6d9ea7c1eb admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:10.795313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff8b57-f70e-4478-894a-5f6d9ea7c1eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:10.795594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7ff8b57-f70e-4478-894a-5f6d9ea7c1eb admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:10.796345 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff8b57-f70e-4478-894a-5f6d9ea7c1eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:10.798342 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7ff8b57-f70e-4478-894a-5f6d9ea7c1eb admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:10.826013 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1151/2298] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:13:10 2022] POST /identity/v3/projects => generated 403 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:13:10.830731 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ff909-6389-417a-afb6-e14322d671ec admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:10.841483 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b92ff909-6389-417a-afb6-e14322d671ec admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:10.900010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:10.915301 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefb5f9-f8ef-47e8-992a-026ded5a798b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:10.916026 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefb5f9-f8ef-47e8-992a-026ded5a798b admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:10.916321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefb5f9-f8ef-47e8-992a-026ded5a798b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:10.916605 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cefb5f9-f8ef-47e8-992a-026ded5a798b admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:10.917311 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefb5f9-f8ef-47e8-992a-026ded5a798b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:10.919266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cefb5f9-f8ef-47e8-992a-026ded5a798b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:10.940731 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1148/2299] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:13:10 2022] POST /identity/v3/users => generated 302 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:13:10.946773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff8b57-f70e-4478-894a-5f6d9ea7c1eb admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:10.957585 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7ff8b57-f70e-4478-894a-5f6d9ea7c1eb admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:10.963199 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981257d-e066-4dbd-9f4b-d69c303c114f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:10.964003 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981257d-e066-4dbd-9f4b-d69c303c114f admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:10.964298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981257d-e066-4dbd-9f4b-d69c303c114f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:10.964588 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9981257d-e066-4dbd-9f4b-d69c303c114f admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:10.965333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981257d-e066-4dbd-9f4b-d69c303c114f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:10.967077 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9981257d-e066-4dbd-9f4b-d69c303c114f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:10.978196 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1152/2300] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:13:10.982144 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefb5f9-f8ef-47e8-992a-026ded5a798b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:10.993160 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cefb5f9-f8ef-47e8-992a-026ded5a798b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:10.999095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74a1fbc-4fe9-4f35-a275-87da06661c98 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:10.999792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a1fbc-4fe9-4f35-a275-87da06661c98 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:11.000116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a1fbc-4fe9-4f35-a275-87da06661c98 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:11.000395 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74a1fbc-4fe9-4f35-a275-87da06661c98 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:11.001252 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a1fbc-4fe9-4f35-a275-87da06661c98 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:11.002871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74a1fbc-4fe9-4f35-a275-87da06661c98 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:11.015046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1149/2301] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:13:11.019935 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981257d-e066-4dbd-9f4b-d69c303c114f admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:11.031100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9981257d-e066-4dbd-9f4b-d69c303c114f admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:11.036277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ce275-9b48-4307-a3d9-2c3121ab191f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:11.037209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ce275-9b48-4307-a3d9-2c3121ab191f admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:11.037504 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ce275-9b48-4307-a3d9-2c3121ab191f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:11.037846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-400ce275-9b48-4307-a3d9-2c3121ab191f admin admin] PATH_INFO: `/v3/projects/33975d4ef96c4c9a984fa7301a585647/users/3cbc33b097c54690885247a69ac118f6/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:11.053843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ce275-9b48-4307-a3d9-2c3121ab191f admin admin] RBAC: Authorizing `identity:create_grant(project_id=33975d4ef96c4c9a984fa7301a585647, user_id=3cbc33b097c54690885247a69ac118f6, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:11.056793 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-400ce275-9b48-4307-a3d9-2c3121ab191f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:11.074281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1153/2302] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:13:11 2022] PUT /identity/v3/projects/33975d4ef96c4c9a984fa7301a585647/users/3cbc33b097c54690885247a69ac118f6/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:11.084671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c5a9b-2300-404e-8d29-e40c54d1a024 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:11.084939 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c5a9b-2300-404e-8d29-e40c54d1a024 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:11.085225 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a93c5a9b-2300-404e-8d29-e40c54d1a024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:11.114872 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a93c5a9b-2300-404e-8d29-e40c54d1a024 None None] MFA Rules not processed for user `3cbc33b097c54690885247a69ac118f6`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:11.116135 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a93c5a9b-2300-404e-8d29-e40c54d1a024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:11.192072 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1150/2303] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:13:11.196853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ce275-9b48-4307-a3d9-2c3121ab191f admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:11.278349 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-400ce275-9b48-4307-a3d9-2c3121ab191f admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:11.285974 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ec17a-98af-40c4-98be-d9146b30e6c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:11.286963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ec17a-98af-40c4-98be-d9146b30e6c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:11.287260 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ec17a-98af-40c4-98be-d9146b30e6c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:11.287547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518ec17a-98af-40c4-98be-d9146b30e6c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:11.288339 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ec17a-98af-40c4-98be-d9146b30e6c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:11.290290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518ec17a-98af-40c4-98be-d9146b30e6c6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:11.329330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1154/2304] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:13:11 2022] POST /identity/v3/projects => generated 405 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:13:11.333149 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c5a9b-2300-404e-8d29-e40c54d1a024 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:11.344394 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a93c5a9b-2300-404e-8d29-e40c54d1a024 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:11.418609 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738da522-5f2b-4ef9-80c0-9a558170d6a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:11.419292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738da522-5f2b-4ef9-80c0-9a558170d6a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:11.419584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738da522-5f2b-4ef9-80c0-9a558170d6a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:11.419891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-738da522-5f2b-4ef9-80c0-9a558170d6a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:11.420587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738da522-5f2b-4ef9-80c0-9a558170d6a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:11.422680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-738da522-5f2b-4ef9-80c0-9a558170d6a2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:11.447043 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1151/2305] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:13:11 2022] POST /identity/v3/users => generated 309 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:13:11.451548 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ec17a-98af-40c4-98be-d9146b30e6c6 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:11.462774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518ec17a-98af-40c4-98be-d9146b30e6c6 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:11.468383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b31d44-2fc2-4018-b972-387fae051b62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:11.469157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b31d44-2fc2-4018-b972-387fae051b62 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:11.469582 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b31d44-2fc2-4018-b972-387fae051b62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:11.472392 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b31d44-2fc2-4018-b972-387fae051b62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:11.475404 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b31d44-2fc2-4018-b972-387fae051b62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:11.475404 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b31d44-2fc2-4018-b972-387fae051b62 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:11.486438 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1155/2306] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:13:11.490781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738da522-5f2b-4ef9-80c0-9a558170d6a2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:11.501134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-738da522-5f2b-4ef9-80c0-9a558170d6a2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:11.506545 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38aacd-3c6d-4b89-a715-04ac78ac8cc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:11.507566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38aacd-3c6d-4b89-a715-04ac78ac8cc1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:11.507932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38aacd-3c6d-4b89-a715-04ac78ac8cc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:11.508297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f38aacd-3c6d-4b89-a715-04ac78ac8cc1 admin admin] PATH_INFO: `/v3/projects/4522124e759c470583526a6bfb822581/users/600a88a313184db88cb2377581bdcf2c/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:11.527658 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f38aacd-3c6d-4b89-a715-04ac78ac8cc1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4522124e759c470583526a6bfb822581, user_id=600a88a313184db88cb2377581bdcf2c, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:11.530619 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f38aacd-3c6d-4b89-a715-04ac78ac8cc1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:11.547864 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1152/2307] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:13:11 2022] PUT /identity/v3/projects/4522124e759c470583526a6bfb822581/users/600a88a313184db88cb2377581bdcf2c/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:11.552540 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b31d44-2fc2-4018-b972-387fae051b62 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:11.568524 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b31d44-2fc2-4018-b972-387fae051b62 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:11.639185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8663ac9-9457-4680-8bf4-d64c9325d83b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:11.640449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8663ac9-9457-4680-8bf4-d64c9325d83b admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:11.640902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8663ac9-9457-4680-8bf4-d64c9325d83b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:11.641243 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8663ac9-9457-4680-8bf4-d64c9325d83b admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:11.641981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8663ac9-9457-4680-8bf4-d64c9325d83b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:11.644064 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8663ac9-9457-4680-8bf4-d64c9325d83b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:11.650582 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:11.656942 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1156/2308] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:13:11.661423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38aacd-3c6d-4b89-a715-04ac78ac8cc1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:11.672028 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f38aacd-3c6d-4b89-a715-04ac78ac8cc1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:11.677137 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27792b-8511-445d-95d4-3dff6d102b48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:11.678037 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27792b-8511-445d-95d4-3dff6d102b48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:11.678328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27792b-8511-445d-95d4-3dff6d102b48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:11.678611 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27792b-8511-445d-95d4-3dff6d102b48 admin admin] PATH_INFO: `/v3/projects/4522124e759c470583526a6bfb822581/users/600a88a313184db88cb2377581bdcf2c/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:11.680720 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27792b-8511-445d-95d4-3dff6d102b48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4522124e759c470583526a6bfb822581, user_id=600a88a313184db88cb2377581bdcf2c, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:11.683681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd27792b-8511-445d-95d4-3dff6d102b48 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:11.696804 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1153/2309] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:13:11 2022] PUT /identity/v3/projects/4522124e759c470583526a6bfb822581/users/600a88a313184db88cb2377581bdcf2c/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:11.709352 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98b62f-3b77-47f5-b250-6eae794450c7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:11.710176 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98b62f-3b77-47f5-b250-6eae794450c7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:11.710555 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf98b62f-3b77-47f5-b250-6eae794450c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:11.741149 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-bf98b62f-3b77-47f5-b250-6eae794450c7 None None] MFA Rules not processed for user `600a88a313184db88cb2377581bdcf2c`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:11.742382 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-bf98b62f-3b77-47f5-b250-6eae794450c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:11.829649 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1157/2310] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:11 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:11.840062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27792b-8511-445d-95d4-3dff6d102b48 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:11.841848 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-fd27792b-8511-445d-95d4-3dff6d102b48 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:11.852722 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:11.903708 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27792b-8511-445d-95d4-3dff6d102b48 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:11.913072 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1a649-dd0c-489f-957f-7c294e98216e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:11.913745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1a649-dd0c-489f-957f-7c294e98216e service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:11.914037 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1a649-dd0c-489f-957f-7c294e98216e service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:11.914322 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1a649-dd0c-489f-957f-7c294e98216e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:11.924192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1a649-dd0c-489f-957f-7c294e98216e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:11.925661 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7f1a649-dd0c-489f-957f-7c294e98216e service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:11.935423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1154/2311] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:11 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:12.085893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98b62f-3b77-47f5-b250-6eae794450c7 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:12.151642 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf98b62f-3b77-47f5-b250-6eae794450c7 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:12.156822 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0ef07-8fc5-46ef-aa42-c54c583cbf60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:12.158101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0ef07-8fc5-46ef-aa42-c54c583cbf60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:12.158389 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0ef07-8fc5-46ef-aa42-c54c583cbf60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:12.158647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b0ef07-8fc5-46ef-aa42-c54c583cbf60 admin admin] PATH_INFO: `/v3/users/3cbc33b097c54690885247a69ac118f6` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:12.159932 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0ef07-8fc5-46ef-aa42-c54c583cbf60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cbc33b097c54690885247a69ac118f6)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:12.162071 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b0ef07-8fc5-46ef-aa42-c54c583cbf60 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:12.211364 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d0b0ef07-8fc5-46ef-aa42-c54c583cbf60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cbc33b097c54690885247a69ac118f6'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:12.212097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d0b0ef07-8fc5-46ef-aa42-c54c583cbf60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cbc33b097c54690885247a69ac118f6'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:12.223844 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d0b0ef07-8fc5-46ef-aa42-c54c583cbf60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cbc33b097c54690885247a69ac118f6'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:12.232847 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d0b0ef07-8fc5-46ef-aa42-c54c583cbf60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cbc33b097c54690885247a69ac118f6'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:12.233559 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d0b0ef07-8fc5-46ef-aa42-c54c583cbf60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cbc33b097c54690885247a69ac118f6'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:12.249911 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1158/2312] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:13:12 2022] DELETE /identity/v3/users/3cbc33b097c54690885247a69ac118f6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:12.252360 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5994f99-3b18-4fb3-9c82-a7956b032522 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:12.252839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5994f99-3b18-4fb3-9c82-a7956b032522 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:12.252937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5994f99-3b18-4fb3-9c82-a7956b032522 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:12.271351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a5994f99-3b18-4fb3-9c82-a7956b032522 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:12.272587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a5994f99-3b18-4fb3-9c82-a7956b032522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:12.332524 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1155/2313] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:12.342798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0ef07-8fc5-46ef-aa42-c54c583cbf60 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:12.344333 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d0b0ef07-8fc5-46ef-aa42-c54c583cbf60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:12.351194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:12.404267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b0ef07-8fc5-46ef-aa42-c54c583cbf60 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:12.408209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f697a2-9a3a-4206-afb7-d68fae56a43a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:12.408867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f697a2-9a3a-4206-afb7-d68fae56a43a service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:12.409155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f697a2-9a3a-4206-afb7-d68fae56a43a service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:12.409436 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f697a2-9a3a-4206-afb7-d68fae56a43a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:12.418521 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f697a2-9a3a-4206-afb7-d68fae56a43a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:12.420058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f697a2-9a3a-4206-afb7-d68fae56a43a service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:12.429797 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1159/2314] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:12.705808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5994f99-3b18-4fb3-9c82-a7956b032522 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:12.707456 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a5994f99-3b18-4fb3-9c82-a7956b032522 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:12.721738 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5994f99-3b18-4fb3-9c82-a7956b032522 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:12.737794 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60d629-e357-4f68-ba63-9ed83a18a6e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:12.739107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60d629-e357-4f68-ba63-9ed83a18a6e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:12.740242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60d629-e357-4f68-ba63-9ed83a18a6e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:12.740642 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be60d629-e357-4f68-ba63-9ed83a18a6e9 admin admin] PATH_INFO: `/v3/projects/33975d4ef96c4c9a984fa7301a585647` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:12.742156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60d629-e357-4f68-ba63-9ed83a18a6e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33975d4ef96c4c9a984fa7301a585647)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:12.744223 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be60d629-e357-4f68-ba63-9ed83a18a6e9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:12.797284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-be60d629-e357-4f68-ba63-9ed83a18a6e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33975d4ef96c4c9a984fa7301a585647', 'request_id': 'req-be60d629-e357-4f68-ba63-9ed83a18a6e9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:12.805774 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-be60d629-e357-4f68-ba63-9ed83a18a6e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33975d4ef96c4c9a984fa7301a585647', 'request_id': 'req-be60d629-e357-4f68-ba63-9ed83a18a6e9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:12.813884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-be60d629-e357-4f68-ba63-9ed83a18a6e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33975d4ef96c4c9a984fa7301a585647', 'request_id': 'req-be60d629-e357-4f68-ba63-9ed83a18a6e9'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:12.825845 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-be60d629-e357-4f68-ba63-9ed83a18a6e9 admin admin] The token cache is being invalidate because project 33975d4ef96c4c9a984fa7301a585647 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:13:12.826134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-be60d629-e357-4f68-ba63-9ed83a18a6e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:12.826838 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-be60d629-e357-4f68-ba63-9ed83a18a6e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:12.829653 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1156/2315] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:13:12 2022] DELETE /identity/v3/projects/33975d4ef96c4c9a984fa7301a585647 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:12.833637 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f697a2-9a3a-4206-afb7-d68fae56a43a service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:12.843490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f697a2-9a3a-4206-afb7-d68fae56a43a service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:12.913156 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46bed23-5e79-4bd3-a8d1-a8c81c9a95a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:12.914135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bed23-5e79-4bd3-a8d1-a8c81c9a95a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:12.914481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bed23-5e79-4bd3-a8d1-a8c81c9a95a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:12.914835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46bed23-5e79-4bd3-a8d1-a8c81c9a95a8 admin admin] PATH_INFO: `/v3/users/600a88a313184db88cb2377581bdcf2c` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:12.916228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bed23-5e79-4bd3-a8d1-a8c81c9a95a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=600a88a313184db88cb2377581bdcf2c)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:12.918136 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46bed23-5e79-4bd3-a8d1-a8c81c9a95a8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:12.972422 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e46bed23-5e79-4bd3-a8d1-a8c81c9a95a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '600a88a313184db88cb2377581bdcf2c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:12.973242 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e46bed23-5e79-4bd3-a8d1-a8c81c9a95a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '600a88a313184db88cb2377581bdcf2c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:12.986505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e46bed23-5e79-4bd3-a8d1-a8c81c9a95a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '600a88a313184db88cb2377581bdcf2c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:12.994963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e46bed23-5e79-4bd3-a8d1-a8c81c9a95a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '600a88a313184db88cb2377581bdcf2c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:12.995705 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-e46bed23-5e79-4bd3-a8d1-a8c81c9a95a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '600a88a313184db88cb2377581bdcf2c'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:13.008753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1160/2316] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:13:12 2022] DELETE /identity/v3/users/600a88a313184db88cb2377581bdcf2c => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:13.396448 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60d629-e357-4f68-ba63-9ed83a18a6e9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:13.398423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-be60d629-e357-4f68-ba63-9ed83a18a6e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:13.412202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be60d629-e357-4f68-ba63-9ed83a18a6e9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:13.447803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:13.474067 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d608f77-39c7-4e8f-9367-4e0672775508 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:13.474946 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d608f77-39c7-4e8f-9367-4e0672775508 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:13.475234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d608f77-39c7-4e8f-9367-4e0672775508 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:13.475516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d608f77-39c7-4e8f-9367-4e0672775508 admin admin] PATH_INFO: `/v3/projects/4522124e759c470583526a6bfb822581` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:13.476752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d608f77-39c7-4e8f-9367-4e0672775508 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4522124e759c470583526a6bfb822581)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:13.478705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d608f77-39c7-4e8f-9367-4e0672775508 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:13.529210 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5d608f77-39c7-4e8f-9367-4e0672775508 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4522124e759c470583526a6bfb822581', 'request_id': 'req-5d608f77-39c7-4e8f-9367-4e0672775508'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:13.538980 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5d608f77-39c7-4e8f-9367-4e0672775508 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4522124e759c470583526a6bfb822581', 'request_id': 'req-5d608f77-39c7-4e8f-9367-4e0672775508'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:13.548476 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5d608f77-39c7-4e8f-9367-4e0672775508 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4522124e759c470583526a6bfb822581', 'request_id': 'req-5d608f77-39c7-4e8f-9367-4e0672775508'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:13.560411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5d608f77-39c7-4e8f-9367-4e0672775508 admin admin] The token cache is being invalidate because project 4522124e759c470583526a6bfb822581 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:13:13.560711 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5d608f77-39c7-4e8f-9367-4e0672775508 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:13.561383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-5d608f77-39c7-4e8f-9367-4e0672775508 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:13.563048 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1157/2317] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:13:13 2022] DELETE /identity/v3/projects/4522124e759c470583526a6bfb822581 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:13.570247 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f29b0a-0214-48e4-a073-c89c58eff63a None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:13.570544 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f29b0a-0214-48e4-a073-c89c58eff63a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:13.570833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f29b0a-0214-48e4-a073-c89c58eff63a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:13.590040 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-84f29b0a-0214-48e4-a073-c89c58eff63a None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:13.591282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-84f29b0a-0214-48e4-a073-c89c58eff63a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:13.613115 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:13.653164 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1161/2318] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:13.661226 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db63d97-e4d6-4bc9-b2db-49358d60a5a7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:13.661527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db63d97-e4d6-4bc9-b2db-49358d60a5a7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:13.661810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db63d97-e4d6-4bc9-b2db-49358d60a5a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:13.681859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-4db63d97-e4d6-4bc9-b2db-49358d60a5a7 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:13.683080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4db63d97-e4d6-4bc9-b2db-49358d60a5a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:13.692784 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1158/2319] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:13.697320 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f29b0a-0214-48e4-a073-c89c58eff63a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:13.707647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f29b0a-0214-48e4-a073-c89c58eff63a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:13.713386 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362872d-cf50-411d-ae6e-3edc8d7d870e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:13.714099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362872d-cf50-411d-ae6e-3edc8d7d870e admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:13.714373 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362872d-cf50-411d-ae6e-3edc8d7d870e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:13.714676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2362872d-cf50-411d-ae6e-3edc8d7d870e admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:13.715420 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362872d-cf50-411d-ae6e-3edc8d7d870e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:13:13.715736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362872d-cf50-411d-ae6e-3edc8d7d870e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:13.717399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2362872d-cf50-411d-ae6e-3edc8d7d870e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:13.729879 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1162/2320] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:13:13.734617 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db63d97-e4d6-4bc9-b2db-49358d60a5a7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:13.744442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db63d97-e4d6-4bc9-b2db-49358d60a5a7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:13.750350 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fc7b9-b7a5-44be-af09-52c974370480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:13.751019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fc7b9-b7a5-44be-af09-52c974370480 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:13.751310 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fc7b9-b7a5-44be-af09-52c974370480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:13.751586 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f0fc7b9-b7a5-44be-af09-52c974370480 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:13.752317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fc7b9-b7a5-44be-af09-52c974370480 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:13.754231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f0fc7b9-b7a5-44be-af09-52c974370480 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:13.785595 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1159/2321] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:13:13 2022] POST /identity/v3/projects => generated 367 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:13:13.789809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362872d-cf50-411d-ae6e-3edc8d7d870e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:13.800137 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2362872d-cf50-411d-ae6e-3edc8d7d870e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:13.856330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c925b-cf21-4cd5-8925-7a5b5c511dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:13.856995 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c925b-cf21-4cd5-8925-7a5b5c511dc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:13.857279 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c925b-cf21-4cd5-8925-7a5b5c511dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:13.857557 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc0c925b-cf21-4cd5-8925-7a5b5c511dc7 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:13.858247 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0c925b-cf21-4cd5-8925-7a5b5c511dc7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:13.860230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc0c925b-cf21-4cd5-8925-7a5b5c511dc7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:13.881343 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1163/2322] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:13:13 2022] POST /identity/v3/users => generated 284 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:13:13.885628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fc7b9-b7a5-44be-af09-52c974370480 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:13.896778 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f0fc7b9-b7a5-44be-af09-52c974370480 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:13.901820 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7a130-1178-4de2-b730-c16027f9e2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:13.902494 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7a130-1178-4de2-b730-c16027f9e2a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:13.902785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7a130-1178-4de2-b730-c16027f9e2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:13.903072 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e7a130-1178-4de2-b730-c16027f9e2a2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:13.903959 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7a130-1178-4de2-b730-c16027f9e2a2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:13.905553 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e7a130-1178-4de2-b730-c16027f9e2a2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:13.917469 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1160/2323] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:13:13.921617 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c925b-cf21-4cd5-8925-7a5b5c511dc7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:13.932053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc0c925b-cf21-4cd5-8925-7a5b5c511dc7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:13.937502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c6658c-86ac-45a2-9b0c-d5c19d92f68c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:13.938221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6658c-86ac-45a2-9b0c-d5c19d92f68c admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:13.938502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6658c-86ac-45a2-9b0c-d5c19d92f68c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:13.938787 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c6658c-86ac-45a2-9b0c-d5c19d92f68c admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:13.939522 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6658c-86ac-45a2-9b0c-d5c19d92f68c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:13.941164 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c6658c-86ac-45a2-9b0c-d5c19d92f68c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:13.951395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1164/2324] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:13:13.956292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7a130-1178-4de2-b730-c16027f9e2a2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:13.967144 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e7a130-1178-4de2-b730-c16027f9e2a2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:13.972016 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e232e3-f122-4334-b603-f2d6d7d5e30c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:13.972906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e232e3-f122-4334-b603-f2d6d7d5e30c admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:13.973196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e232e3-f122-4334-b603-f2d6d7d5e30c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:13.973478 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e232e3-f122-4334-b603-f2d6d7d5e30c admin admin] PATH_INFO: `/v3/projects/4badeb5129674ea893101d20e54b1bec/users/a23df11da6bc47dc8fbe6b3d8ef447e8/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:13.990123 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e232e3-f122-4334-b603-f2d6d7d5e30c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4badeb5129674ea893101d20e54b1bec, user_id=a23df11da6bc47dc8fbe6b3d8ef447e8, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:13.992903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e232e3-f122-4334-b603-f2d6d7d5e30c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:14.006706 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1161/2325] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:13:13 2022] PUT /identity/v3/projects/4badeb5129674ea893101d20e54b1bec/users/a23df11da6bc47dc8fbe6b3d8ef447e8/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:14.017230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a88e11-36db-40a7-97dc-a02107aebe28 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:14.017530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a88e11-36db-40a7-97dc-a02107aebe28 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:14.017813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a88e11-36db-40a7-97dc-a02107aebe28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:14.045603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-98a88e11-36db-40a7-97dc-a02107aebe28 None None] MFA Rules not processed for user `a23df11da6bc47dc8fbe6b3d8ef447e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:14.046821 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-98a88e11-36db-40a7-97dc-a02107aebe28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:14.110077 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1165/2326] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:14 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:14.114846 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e232e3-f122-4334-b603-f2d6d7d5e30c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:14.181470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e232e3-f122-4334-b603-f2d6d7d5e30c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:14.186735 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087e6e6-b628-4114-8026-cacee3302b47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:14.187401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087e6e6-b628-4114-8026-cacee3302b47 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:14.187696 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087e6e6-b628-4114-8026-cacee3302b47 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:14.188050 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5087e6e6-b628-4114-8026-cacee3302b47 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:14.188742 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087e6e6-b628-4114-8026-cacee3302b47 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:14.190714 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5087e6e6-b628-4114-8026-cacee3302b47 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:14.194826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:14.219718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1162/2327] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:13:14 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:13:14.224524 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a88e11-36db-40a7-97dc-a02107aebe28 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:14.239340 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a88e11-36db-40a7-97dc-a02107aebe28 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:14.260345 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:14.304057 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaecbab-21d1-4730-97ea-b707af252729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:14.305919 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaecbab-21d1-4730-97ea-b707af252729 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:14.305919 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaecbab-21d1-4730-97ea-b707af252729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:14.305919 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aaecbab-21d1-4730-97ea-b707af252729 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:14.306475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaecbab-21d1-4730-97ea-b707af252729 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:14.308577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3aaecbab-21d1-4730-97ea-b707af252729 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:14.332269 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1166/2328] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:13:14 2022] POST /identity/v3/users => generated 290 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:13:14.333548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087e6e6-b628-4114-8026-cacee3302b47 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:14.344668 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5087e6e6-b628-4114-8026-cacee3302b47 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:14.351922 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99776418-c86e-47d1-8c49-3915976ccae5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:14.352655 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99776418-c86e-47d1-8c49-3915976ccae5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:14.352944 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99776418-c86e-47d1-8c49-3915976ccae5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:14.353220 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99776418-c86e-47d1-8c49-3915976ccae5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:14.354011 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99776418-c86e-47d1-8c49-3915976ccae5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:14.355622 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99776418-c86e-47d1-8c49-3915976ccae5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:14.371385 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1163/2329] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:13:14.375691 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaecbab-21d1-4730-97ea-b707af252729 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:14.386461 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aaecbab-21d1-4730-97ea-b707af252729 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:14.391236 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebd252-ec95-4c1a-a2b7-3d2e44d752a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:14.392175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebd252-ec95-4c1a-a2b7-3d2e44d752a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:14.392467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebd252-ec95-4c1a-a2b7-3d2e44d752a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:14.392753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ebd252-ec95-4c1a-a2b7-3d2e44d752a7 admin admin] PATH_INFO: `/v3/projects/59e5e276ff92429e824c9c364d1f3519/users/81a2ac5b26f3461cb9a5674a5147e610/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:14.408553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebd252-ec95-4c1a-a2b7-3d2e44d752a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59e5e276ff92429e824c9c364d1f3519, user_id=81a2ac5b26f3461cb9a5674a5147e610, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:14.411198 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ebd252-ec95-4c1a-a2b7-3d2e44d752a7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:14.425897 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1167/2330] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:13:14 2022] PUT /identity/v3/projects/59e5e276ff92429e824c9c364d1f3519/users/81a2ac5b26f3461cb9a5674a5147e610/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:13:14.429975 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99776418-c86e-47d1-8c49-3915976ccae5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:14.440208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99776418-c86e-47d1-8c49-3915976ccae5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:14.499490 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081119b9-d4eb-403f-bf36-f2f750eb21a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:14.500221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081119b9-d4eb-403f-bf36-f2f750eb21a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:14.500537 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081119b9-d4eb-403f-bf36-f2f750eb21a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:14.500819 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081119b9-d4eb-403f-bf36-f2f750eb21a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:14.501635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081119b9-d4eb-403f-bf36-f2f750eb21a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:14.503282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081119b9-d4eb-403f-bf36-f2f750eb21a5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:14.513877 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1164/2331] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:13:14.518183 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebd252-ec95-4c1a-a2b7-3d2e44d752a7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:14.529933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ebd252-ec95-4c1a-a2b7-3d2e44d752a7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:14.534739 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac6f596-8c8f-467c-948d-e9ee883accde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:14.535638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6f596-8c8f-467c-948d-e9ee883accde admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:14.535967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6f596-8c8f-467c-948d-e9ee883accde admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:14.536255 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aac6f596-8c8f-467c-948d-e9ee883accde admin admin] PATH_INFO: `/v3/projects/59e5e276ff92429e824c9c364d1f3519/users/81a2ac5b26f3461cb9a5674a5147e610/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:14.538353 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac6f596-8c8f-467c-948d-e9ee883accde admin admin] RBAC: Authorizing `identity:create_grant(project_id=59e5e276ff92429e824c9c364d1f3519, user_id=81a2ac5b26f3461cb9a5674a5147e610, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:14.541220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aac6f596-8c8f-467c-948d-e9ee883accde admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:14.551710 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1168/2332] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:13:14 2022] PUT /identity/v3/projects/59e5e276ff92429e824c9c364d1f3519/users/81a2ac5b26f3461cb9a5674a5147e610/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:13:14.564704 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff215e-2e1f-45a0-84da-6a74a412bc2d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:14.565005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff215e-2e1f-45a0-84da-6a74a412bc2d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:14.565281 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ff215e-2e1f-45a0-84da-6a74a412bc2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:14.591768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-34ff215e-2e1f-45a0-84da-6a74a412bc2d None None] MFA Rules not processed for user `81a2ac5b26f3461cb9a5674a5147e610`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:14.593024 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-34ff215e-2e1f-45a0-84da-6a74a412bc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:14.669780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1165/2333] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:14 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:13:14.681670 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac6f596-8c8f-467c-948d-e9ee883accde admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:14.683503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-aac6f596-8c8f-467c-948d-e9ee883accde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:14.707813 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff215e-2e1f-45a0-84da-6a74a412bc2d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:14.709296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-34ff215e-2e1f-45a0-84da-6a74a412bc2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:14.749765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aac6f596-8c8f-467c-948d-e9ee883accde admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:14.754093 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2648d-186a-4b59-9cfc-2470ca265ba4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:14.754755 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2648d-186a-4b59-9cfc-2470ca265ba4 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:14.755071 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2648d-186a-4b59-9cfc-2470ca265ba4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:14.755357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c2648d-186a-4b59-9cfc-2470ca265ba4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:14.765140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2648d-186a-4b59-9cfc-2470ca265ba4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:14.766585 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c2648d-186a-4b59-9cfc-2470ca265ba4 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:14.768319 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ff215e-2e1f-45a0-84da-6a74a412bc2d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:14.771991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5bcb1-e2cc-4c65-ab1e-9da02b3d465b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:14.772646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5bcb1-e2cc-4c65-ab1e-9da02b3d465b service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:14.772936 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5bcb1-e2cc-4c65-ab1e-9da02b3d465b service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:14.773218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5bcb1-e2cc-4c65-ab1e-9da02b3d465b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:14.774985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6fb5bcb1-e2cc-4c65-ab1e-9da02b3d465b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:14.776134 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1169/2334] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:14 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:14.785287 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5bcb1-e2cc-4c65-ab1e-9da02b3d465b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:14.786726 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5bcb1-e2cc-4c65-ab1e-9da02b3d465b service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:14.791153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:14.796298 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1166/2335] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:14 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:14.861258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2648d-186a-4b59-9cfc-2470ca265ba4 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:14.869840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c2648d-186a-4b59-9cfc-2470ca265ba4 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:14.873688 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef009391-1513-4dda-9adb-4de00215dbce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:14.874348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef009391-1513-4dda-9adb-4de00215dbce service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:14.874629 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef009391-1513-4dda-9adb-4de00215dbce service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:14.874903 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef009391-1513-4dda-9adb-4de00215dbce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:14.935078 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef009391-1513-4dda-9adb-4de00215dbce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:14.936476 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef009391-1513-4dda-9adb-4de00215dbce service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:14.945679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1170/2336] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:14 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:19.251500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5bcb1-e2cc-4c65-ab1e-9da02b3d465b service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:19.253583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6fb5bcb1-e2cc-4c65-ab1e-9da02b3d465b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:19.350763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5bcb1-e2cc-4c65-ab1e-9da02b3d465b service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:19.356628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80f8db3-118c-411f-9d96-d48cdd152a80 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:19.357377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8db3-118c-411f-9d96-d48cdd152a80 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:19.357732 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8db3-118c-411f-9d96-d48cdd152a80 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:19.358185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80f8db3-118c-411f-9d96-d48cdd152a80 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:19.360139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f80f8db3-118c-411f-9d96-d48cdd152a80 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:19.439333 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f8db3-118c-411f-9d96-d48cdd152a80 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:19.441676 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f80f8db3-118c-411f-9d96-d48cdd152a80 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:19.463762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1167/2337] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:19 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 212 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:19.919303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef009391-1513-4dda-9adb-4de00215dbce service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:19.937369 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:19.982799 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef009391-1513-4dda-9adb-4de00215dbce service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:19.987366 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f466044-ae0c-4cbc-9042-21f71c3296c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:19.988313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f466044-ae0c-4cbc-9042-21f71c3296c6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:19.988602 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f466044-ae0c-4cbc-9042-21f71c3296c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:19.988885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f466044-ae0c-4cbc-9042-21f71c3296c6 admin admin] PATH_INFO: `/v3/users/a23df11da6bc47dc8fbe6b3d8ef447e8` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:19.990082 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f466044-ae0c-4cbc-9042-21f71c3296c6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a23df11da6bc47dc8fbe6b3d8ef447e8)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:19.992048 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f466044-ae0c-4cbc-9042-21f71c3296c6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:20.041853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9f466044-ae0c-4cbc-9042-21f71c3296c6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a23df11da6bc47dc8fbe6b3d8ef447e8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:20.042514 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9f466044-ae0c-4cbc-9042-21f71c3296c6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a23df11da6bc47dc8fbe6b3d8ef447e8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:20.054447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9f466044-ae0c-4cbc-9042-21f71c3296c6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a23df11da6bc47dc8fbe6b3d8ef447e8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:20.062888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9f466044-ae0c-4cbc-9042-21f71c3296c6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a23df11da6bc47dc8fbe6b3d8ef447e8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:20.063592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9f466044-ae0c-4cbc-9042-21f71c3296c6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a23df11da6bc47dc8fbe6b3d8ef447e8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:20.078292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1171/2338] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:13:19 2022] DELETE /identity/v3/users/a23df11da6bc47dc8fbe6b3d8ef447e8 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:20.082957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee3ae3-3d55-4b3c-af01-f52bcd96c5a1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:20.083260 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee3ae3-3d55-4b3c-af01-f52bcd96c5a1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:20.083579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee3ae3-3d55-4b3c-af01-f52bcd96c5a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:20.102674 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-efee3ae3-3d55-4b3c-af01-f52bcd96c5a1 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:20.103976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-efee3ae3-3d55-4b3c-af01-f52bcd96c5a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:20.167516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1168/2339] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:20.178843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f466044-ae0c-4cbc-9042-21f71c3296c6 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:20.180609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9f466044-ae0c-4cbc-9042-21f71c3296c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:20.216881 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090d0d1-cbba-44c3-a58d-26e4a06bc228 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:20.217177 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090d0d1-cbba-44c3-a58d-26e4a06bc228 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:20.217460 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a090d0d1-cbba-44c3-a58d-26e4a06bc228 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:20.236248 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a090d0d1-cbba-44c3-a58d-26e4a06bc228 None None] MFA Rules not processed for user `708fdc537e1747d7bd5917ed7af9c368`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:20.237538 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a090d0d1-cbba-44c3-a58d-26e4a06bc228 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:20.245782 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f466044-ae0c-4cbc-9042-21f71c3296c6 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:20.249594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df87300-dd81-4df1-bd54-a43759527a4d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:20.250468 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df87300-dd81-4df1-bd54-a43759527a4d service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:20.250741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df87300-dd81-4df1-bd54-a43759527a4d service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:20.251026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df87300-dd81-4df1-bd54-a43759527a4d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:20.261287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df87300-dd81-4df1-bd54-a43759527a4d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:20.262850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df87300-dd81-4df1-bd54-a43759527a4d service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:20.272028 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:20.273164 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1172/2340] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:20.297378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1169/2341] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:20 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:20.578864 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df87300-dd81-4df1-bd54-a43759527a4d service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:20.580624 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3df87300-dd81-4df1-bd54-a43759527a4d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:20.595132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df87300-dd81-4df1-bd54-a43759527a4d service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:20.600178 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5a43b-15e3-42fb-a44b-f6545e33adaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:20.601064 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5a43b-15e3-42fb-a44b-f6545e33adaa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:20.601344 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5a43b-15e3-42fb-a44b-f6545e33adaa admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:20.601636 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d5a43b-15e3-42fb-a44b-f6545e33adaa admin admin] PATH_INFO: `/v3/projects/4badeb5129674ea893101d20e54b1bec` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:20.602876 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5a43b-15e3-42fb-a44b-f6545e33adaa admin admin] RBAC: Authorizing `identity:delete_project(project_id=4badeb5129674ea893101d20e54b1bec)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:20.604879 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d5a43b-15e3-42fb-a44b-f6545e33adaa admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:20.641907 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:20.655590 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-75d5a43b-15e3-42fb-a44b-f6545e33adaa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4badeb5129674ea893101d20e54b1bec', 'request_id': 'req-75d5a43b-15e3-42fb-a44b-f6545e33adaa'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:20.661934 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-75d5a43b-15e3-42fb-a44b-f6545e33adaa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4badeb5129674ea893101d20e54b1bec', 'request_id': 'req-75d5a43b-15e3-42fb-a44b-f6545e33adaa'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:20.670175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-75d5a43b-15e3-42fb-a44b-f6545e33adaa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4badeb5129674ea893101d20e54b1bec', 'request_id': 'req-75d5a43b-15e3-42fb-a44b-f6545e33adaa'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:20.680618 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090d0d1-cbba-44c3-a58d-26e4a06bc228 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:20.681491 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-75d5a43b-15e3-42fb-a44b-f6545e33adaa admin admin] The token cache is being invalidate because project 4badeb5129674ea893101d20e54b1bec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:13:20.681778 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-75d5a43b-15e3-42fb-a44b-f6545e33adaa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:20.682437 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-75d5a43b-15e3-42fb-a44b-f6545e33adaa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:20.684034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1173/2342] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:13:20 2022] DELETE /identity/v3/projects/4badeb5129674ea893101d20e54b1bec => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:20.687777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5a43b-15e3-42fb-a44b-f6545e33adaa admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:20.689098 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-75d5a43b-15e3-42fb-a44b-f6545e33adaa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:20.692326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a090d0d1-cbba-44c3-a58d-26e4a06bc228 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:20.741722 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f771ef6-6f9c-475c-a59a-ecaf8cb36cff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:20.742379 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f771ef6-6f9c-475c-a59a-ecaf8cb36cff service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:20.742667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f771ef6-6f9c-475c-a59a-ecaf8cb36cff service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:20.742949 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f771ef6-6f9c-475c-a59a-ecaf8cb36cff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:20.744782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-2f771ef6-6f9c-475c-a59a-ecaf8cb36cff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:20.754632 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d5a43b-15e3-42fb-a44b-f6545e33adaa admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:20.754839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f771ef6-6f9c-475c-a59a-ecaf8cb36cff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:20.756132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f771ef6-6f9c-475c-a59a-ecaf8cb36cff service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:20.759399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ab7fae-3416-42b2-9405-62963a6a50ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:20.760675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab7fae-3416-42b2-9405-62963a6a50ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:20.760962 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab7fae-3416-42b2-9405-62963a6a50ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:20.761239 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ab7fae-3416-42b2-9405-62963a6a50ec admin admin] PATH_INFO: `/v3/users/81a2ac5b26f3461cb9a5674a5147e610` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:20.762485 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab7fae-3416-42b2-9405-62963a6a50ec admin admin] RBAC: Authorizing `identity:delete_user(user_id=81a2ac5b26f3461cb9a5674a5147e610)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:20.764482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ab7fae-3416-42b2-9405-62963a6a50ec admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:20.764838 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1170/2343] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:13:20.811202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-52ab7fae-3416-42b2-9405-62963a6a50ec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81a2ac5b26f3461cb9a5674a5147e610'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:20.811906 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-52ab7fae-3416-42b2-9405-62963a6a50ec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81a2ac5b26f3461cb9a5674a5147e610'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:20.825744 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-52ab7fae-3416-42b2-9405-62963a6a50ec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81a2ac5b26f3461cb9a5674a5147e610'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:20.833959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-52ab7fae-3416-42b2-9405-62963a6a50ec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81a2ac5b26f3461cb9a5674a5147e610'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:20.834601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-52ab7fae-3416-42b2-9405-62963a6a50ec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81a2ac5b26f3461cb9a5674a5147e610'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:20.849643 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1174/2344] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:13:20 2022] DELETE /identity/v3/users/81a2ac5b26f3461cb9a5674a5147e610 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:13:21.205358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f771ef6-6f9c-475c-a59a-ecaf8cb36cff service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:21.218350 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f771ef6-6f9c-475c-a59a-ecaf8cb36cff service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:21.291134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ec70e-f868-431d-b1f6-de83e59709a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:21.292088 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ec70e-f868-431d-b1f6-de83e59709a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:21.292377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ec70e-f868-431d-b1f6-de83e59709a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:21.292660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ec70e-f868-431d-b1f6-de83e59709a3 admin admin] PATH_INFO: `/v3/projects/59e5e276ff92429e824c9c364d1f3519` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:21.293866 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ec70e-f868-431d-b1f6-de83e59709a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=59e5e276ff92429e824c9c364d1f3519)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:21.296057 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413ec70e-f868-431d-b1f6-de83e59709a3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:21.347398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-413ec70e-f868-431d-b1f6-de83e59709a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59e5e276ff92429e824c9c364d1f3519', 'request_id': 'req-413ec70e-f868-431d-b1f6-de83e59709a3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:21.356680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-413ec70e-f868-431d-b1f6-de83e59709a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59e5e276ff92429e824c9c364d1f3519', 'request_id': 'req-413ec70e-f868-431d-b1f6-de83e59709a3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:21.363528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-413ec70e-f868-431d-b1f6-de83e59709a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59e5e276ff92429e824c9c364d1f3519', 'request_id': 'req-413ec70e-f868-431d-b1f6-de83e59709a3'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:21.375653 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-413ec70e-f868-431d-b1f6-de83e59709a3 admin admin] The token cache is being invalidate because project 59e5e276ff92429e824c9c364d1f3519 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:13:21.376125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-413ec70e-f868-431d-b1f6-de83e59709a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:21.376634 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-413ec70e-f868-431d-b1f6-de83e59709a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:21.381563 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1171/2345] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:13:21 2022] DELETE /identity/v3/projects/59e5e276ff92429e824c9c364d1f3519 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:21.383525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d980a-c127-464e-b0e2-e2c084c6fbcc None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:21.384055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d980a-c127-464e-b0e2-e2c084c6fbcc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:21.384272 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437d980a-c127-464e-b0e2-e2c084c6fbcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:21.404049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-437d980a-c127-464e-b0e2-e2c084c6fbcc None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:21.405496 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-437d980a-c127-464e-b0e2-e2c084c6fbcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:21.443282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:21.478795 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1175/2346] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:21.486733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066d30d-dd52-434a-bfe3-13163073bbee None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:21.487027 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066d30d-dd52-434a-bfe3-13163073bbee None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:21.487308 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f066d30d-dd52-434a-bfe3-13163073bbee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:21.512973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f066d30d-dd52-434a-bfe3-13163073bbee None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:21.514245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f066d30d-dd52-434a-bfe3-13163073bbee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:21.522227 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1172/2347] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:21.526573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d980a-c127-464e-b0e2-e2c084c6fbcc None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:21.537297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437d980a-c127-464e-b0e2-e2c084c6fbcc None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:21.543104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d516d78-5459-4b32-89ef-07f2c061cf84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:21.543784 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d516d78-5459-4b32-89ef-07f2c061cf84 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:21.544117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d516d78-5459-4b32-89ef-07f2c061cf84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:21.544402 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d516d78-5459-4b32-89ef-07f2c061cf84 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:21.545145 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d516d78-5459-4b32-89ef-07f2c061cf84 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:13:21.545463 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d516d78-5459-4b32-89ef-07f2c061cf84 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:21.547213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d516d78-5459-4b32-89ef-07f2c061cf84 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:21.561457 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1176/2348] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:13:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:13:21.564733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066d30d-dd52-434a-bfe3-13163073bbee None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:21.574463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f066d30d-dd52-434a-bfe3-13163073bbee None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:21.579445 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420b104-8224-4066-964a-ee052d71ec3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:21.580156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420b104-8224-4066-964a-ee052d71ec3b admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:21.580448 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420b104-8224-4066-964a-ee052d71ec3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:21.580728 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6420b104-8224-4066-964a-ee052d71ec3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:21.581429 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420b104-8224-4066-964a-ee052d71ec3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:21.583515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6420b104-8224-4066-964a-ee052d71ec3b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:21.588744 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:21.613914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1173/2349] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:13:21 2022] POST /identity/v3/projects => generated 383 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:13:21.618188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d516d78-5459-4b32-89ef-07f2c061cf84 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:21.629885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d516d78-5459-4b32-89ef-07f2c061cf84 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:21.698464 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ad0a5-1a9c-4b9e-9a84-a66b25bda21c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:21.699408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ad0a5-1a9c-4b9e-9a84-a66b25bda21c admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:21.699885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ad0a5-1a9c-4b9e-9a84-a66b25bda21c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:21.700325 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87ad0a5-1a9c-4b9e-9a84-a66b25bda21c admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:21.701266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ad0a5-1a9c-4b9e-9a84-a66b25bda21c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:21.703745 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87ad0a5-1a9c-4b9e-9a84-a66b25bda21c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:21.732793 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1177/2350] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:13:21 2022] POST /identity/v3/users => generated 292 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:13:21.734086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420b104-8224-4066-964a-ee052d71ec3b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:21.744898 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6420b104-8224-4066-964a-ee052d71ec3b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:21.749787 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2662a79-8289-474d-a9ef-df4d3a71d690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:21.750561 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2662a79-8289-474d-a9ef-df4d3a71d690 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:21.750935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2662a79-8289-474d-a9ef-df4d3a71d690 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:21.751278 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2662a79-8289-474d-a9ef-df4d3a71d690 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:21.752109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2662a79-8289-474d-a9ef-df4d3a71d690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:21.753856 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2662a79-8289-474d-a9ef-df4d3a71d690 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:21.767872 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1174/2351] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:13:21.769652 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ad0a5-1a9c-4b9e-9a84-a66b25bda21c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:21.780486 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87ad0a5-1a9c-4b9e-9a84-a66b25bda21c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:21.785198 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172f6971-1b17-4555-b0e7-5bf716d5ba4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:21.785874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f6971-1b17-4555-b0e7-5bf716d5ba4e admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:21.786162 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f6971-1b17-4555-b0e7-5bf716d5ba4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:21.786441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172f6971-1b17-4555-b0e7-5bf716d5ba4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:21.787201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f6971-1b17-4555-b0e7-5bf716d5ba4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:21.788850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172f6971-1b17-4555-b0e7-5bf716d5ba4e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:21.803277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1178/2352] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:21 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:13:21.805056 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2662a79-8289-474d-a9ef-df4d3a71d690 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:21.816358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2662a79-8289-474d-a9ef-df4d3a71d690 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:21.821631 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d5231-d9dd-462e-834f-c9c29ae2b5fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:21.822730 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d5231-d9dd-462e-834f-c9c29ae2b5fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:21.823018 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d5231-d9dd-462e-834f-c9c29ae2b5fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:21.823296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2d5231-d9dd-462e-834f-c9c29ae2b5fd admin admin] PATH_INFO: `/v3/projects/b6855f7ea31e4e9983b26db3d34c03fb/users/c839e107cc264600a04b675e76714c87/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:21.842461 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d5231-d9dd-462e-834f-c9c29ae2b5fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=b6855f7ea31e4e9983b26db3d34c03fb, user_id=c839e107cc264600a04b675e76714c87, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:21.845579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2d5231-d9dd-462e-834f-c9c29ae2b5fd admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:21.862234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1175/2353] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:13:21 2022] PUT /identity/v3/projects/b6855f7ea31e4e9983b26db3d34c03fb/users/c839e107cc264600a04b675e76714c87/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:21.872888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7098cff-58c3-42e5-8830-ba22360d1542 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:21.873196 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7098cff-58c3-42e5-8830-ba22360d1542 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:21.873461 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7098cff-58c3-42e5-8830-ba22360d1542 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:21.902122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f7098cff-58c3-42e5-8830-ba22360d1542 None None] MFA Rules not processed for user `c839e107cc264600a04b675e76714c87`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:21.903341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f7098cff-58c3-42e5-8830-ba22360d1542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:21.975546 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1179/2354] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:21 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:21.986600 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d5231-d9dd-462e-834f-c9c29ae2b5fd admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:21.988580 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3c2d5231-d9dd-462e-834f-c9c29ae2b5fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:22.049686 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2d5231-d9dd-462e-834f-c9c29ae2b5fd admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:22.053418 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6313eb6-8b3d-46d1-ad87-32633476a012 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:22.054194 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6313eb6-8b3d-46d1-ad87-32633476a012 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:22.054485 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6313eb6-8b3d-46d1-ad87-32633476a012 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:22.054785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6313eb6-8b3d-46d1-ad87-32633476a012 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:22.063567 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6313eb6-8b3d-46d1-ad87-32633476a012 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:22.065071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6313eb6-8b3d-46d1-ad87-32633476a012 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:22.074224 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1176/2355] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:23.750648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7098cff-58c3-42e5-8830-ba22360d1542 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:23.753053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f7098cff-58c3-42e5-8830-ba22360d1542 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:23.808618 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:23.853997 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7098cff-58c3-42e5-8830-ba22360d1542 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:23.859666 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588fb241-e2e6-4f6d-afcb-785075607ae8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:23.860439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fb241-e2e6-4f6d-afcb-785075607ae8 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:23.860732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fb241-e2e6-4f6d-afcb-785075607ae8 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:23.861029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588fb241-e2e6-4f6d-afcb-785075607ae8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:23.862977 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-588fb241-e2e6-4f6d-afcb-785075607ae8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:23.876152 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588fb241-e2e6-4f6d-afcb-785075607ae8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:23.877954 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588fb241-e2e6-4f6d-afcb-785075607ae8 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:23.890159 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1180/2356] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:23 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:30.663189 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6313eb6-8b3d-46d1-ad87-32633476a012 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:30.726691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6313eb6-8b3d-46d1-ad87-32633476a012 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:30.732284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6dec0-f8ec-41cc-8196-caff480fa050 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:30.733154 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6dec0-f8ec-41cc-8196-caff480fa050 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:30.733443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6dec0-f8ec-41cc-8196-caff480fa050 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:30.733727 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef6dec0-f8ec-41cc-8196-caff480fa050 admin admin] PATH_INFO: `/v3/users/c839e107cc264600a04b675e76714c87` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:30.734985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef6dec0-f8ec-41cc-8196-caff480fa050 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c839e107cc264600a04b675e76714c87)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:30.736960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ef6dec0-f8ec-41cc-8196-caff480fa050 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:30.796418 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3ef6dec0-f8ec-41cc-8196-caff480fa050 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c839e107cc264600a04b675e76714c87'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:30.803542 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:30.809602 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3ef6dec0-f8ec-41cc-8196-caff480fa050 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c839e107cc264600a04b675e76714c87'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:30.817401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3ef6dec0-f8ec-41cc-8196-caff480fa050 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c839e107cc264600a04b675e76714c87'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:30.818016 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3ef6dec0-f8ec-41cc-8196-caff480fa050 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c839e107cc264600a04b675e76714c87'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:30.825161 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3ef6dec0-f8ec-41cc-8196-caff480fa050 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c839e107cc264600a04b675e76714c87'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:30.831275 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1177/2357] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:13:30 2022] DELETE /identity/v3/users/c839e107cc264600a04b675e76714c87 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:30.835885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9b18f-b1a9-437c-b51f-314582284310 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:30.836394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9b18f-b1a9-437c-b51f-314582284310 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:30.836503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf9b18f-b1a9-437c-b51f-314582284310 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:30.854601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-baf9b18f-b1a9-437c-b51f-314582284310 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:30.855928 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-baf9b18f-b1a9-437c-b51f-314582284310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:30.915453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1181/2358] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:30.925218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6dec0-f8ec-41cc-8196-caff480fa050 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:30.926915 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3ef6dec0-f8ec-41cc-8196-caff480fa050 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:30.986071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef6dec0-f8ec-41cc-8196-caff480fa050 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:30.990433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c9810-f676-4622-9cc2-3ac5a4687037 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:30.991085 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c9810-f676-4622-9cc2-3ac5a4687037 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:30.991374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c9810-f676-4622-9cc2-3ac5a4687037 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:30.991660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8c9810-f676-4622-9cc2-3ac5a4687037 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:31.001180 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c9810-f676-4622-9cc2-3ac5a4687037 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:31.002626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8c9810-f676-4622-9cc2-3ac5a4687037 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:31.012784 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1178/2359] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:31.292271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9b18f-b1a9-437c-b51f-314582284310 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:31.294676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-baf9b18f-b1a9-437c-b51f-314582284310 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:31.309741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf9b18f-b1a9-437c-b51f-314582284310 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:31.314657 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb6252d-af9d-4384-8270-0cc7969ca8a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:31.315572 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6252d-af9d-4384-8270-0cc7969ca8a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:31.315897 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6252d-af9d-4384-8270-0cc7969ca8a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:31.316193 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb6252d-af9d-4384-8270-0cc7969ca8a2 admin admin] PATH_INFO: `/v3/projects/b6855f7ea31e4e9983b26db3d34c03fb` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:31.317403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6252d-af9d-4384-8270-0cc7969ca8a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b6855f7ea31e4e9983b26db3d34c03fb)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:31.319427 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb6252d-af9d-4384-8270-0cc7969ca8a2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:31.371508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-4eb6252d-af9d-4384-8270-0cc7969ca8a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b6855f7ea31e4e9983b26db3d34c03fb', 'request_id': 'req-4eb6252d-af9d-4384-8270-0cc7969ca8a2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:31.377423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-4eb6252d-af9d-4384-8270-0cc7969ca8a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b6855f7ea31e4e9983b26db3d34c03fb', 'request_id': 'req-4eb6252d-af9d-4384-8270-0cc7969ca8a2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:31.386293 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-4eb6252d-af9d-4384-8270-0cc7969ca8a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b6855f7ea31e4e9983b26db3d34c03fb', 'request_id': 'req-4eb6252d-af9d-4384-8270-0cc7969ca8a2'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:31.396941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c9810-f676-4622-9cc2-3ac5a4687037 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:31.397931 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5c8c9810-f676-4622-9cc2-3ac5a4687037 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:31.409296 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-4eb6252d-af9d-4384-8270-0cc7969ca8a2 admin admin] The token cache is being invalidate because project b6855f7ea31e4e9983b26db3d34c03fb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:13:31.409623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-4eb6252d-af9d-4384-8270-0cc7969ca8a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:31.410297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-4eb6252d-af9d-4384-8270-0cc7969ca8a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:31.413220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1182/2360] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:13:31 2022] DELETE /identity/v3/projects/b6855f7ea31e4e9983b26db3d34c03fb => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:31.418255 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc940c3-e465-4a5a-be33-5804883ffa01 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:31.418595 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc940c3-e465-4a5a-be33-5804883ffa01 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:31.418930 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc940c3-e465-4a5a-be33-5804883ffa01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:31.438724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-dfc940c3-e465-4a5a-be33-5804883ffa01 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:31.439998 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-dfc940c3-e465-4a5a-be33-5804883ffa01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:31.486255 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8c9810-f676-4622-9cc2-3ac5a4687037 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:31.489826 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:31.514679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1183/2361] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:31.522335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3db6a7-7df4-42c5-af77-9976a81e9b75 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:31.522814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3db6a7-7df4-42c5-af77-9976a81e9b75 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:31.523126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba3db6a7-7df4-42c5-af77-9976a81e9b75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:31.547020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-ba3db6a7-7df4-42c5-af77-9976a81e9b75 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:31.548275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ba3db6a7-7df4-42c5-af77-9976a81e9b75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:31.556354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1184/2362] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:13:31.560509 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3db6a7-7df4-42c5-af77-9976a81e9b75 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:31.569633 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba3db6a7-7df4-42c5-af77-9976a81e9b75 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:31.570685 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ebcdd-4372-4fc7-bd48-866967d46696 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:31.571446 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ebcdd-4372-4fc7-bd48-866967d46696 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:31.571742 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ebcdd-4372-4fc7-bd48-866967d46696 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:31.572068 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354ebcdd-4372-4fc7-bd48-866967d46696 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:31.574074 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-354ebcdd-4372-4fc7-bd48-866967d46696 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:31.574459 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e51fe-96d3-44f0-ae3f-1160b5fe10d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:31.575125 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e51fe-96d3-44f0-ae3f-1160b5fe10d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:31.575415 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e51fe-96d3-44f0-ae3f-1160b5fe10d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:31.575697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32e51fe-96d3-44f0-ae3f-1160b5fe10d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:31.577130 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e51fe-96d3-44f0-ae3f-1160b5fe10d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:13:31.577130 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e51fe-96d3-44f0-ae3f-1160b5fe10d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:31.578537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32e51fe-96d3-44f0-ae3f-1160b5fe10d5 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:31.587769 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ebcdd-4372-4fc7-bd48-866967d46696 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:31.589418 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354ebcdd-4372-4fc7-bd48-866967d46696 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:31.596179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:DELETE=2,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:31.596734 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1185/2363] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:13:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:13:31.598104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e51fe-96d3-44f0-ae3f-1160b5fe10d5 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:31.608287 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1179/2364] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:31.609085 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32e51fe-96d3-44f0-ae3f-1160b5fe10d5 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:31.618232 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f9f0f-8d35-4474-84aa-24878bf1df5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:31.618913 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f9f0f-8d35-4474-84aa-24878bf1df5e admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:31.619206 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f9f0f-8d35-4474-84aa-24878bf1df5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:31.619488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-791f9f0f-8d35-4474-84aa-24878bf1df5e admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:31.620230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f9f0f-8d35-4474-84aa-24878bf1df5e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:31.622344 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-791f9f0f-8d35-4474-84aa-24878bf1df5e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:31.656779 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1186/2365] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:13:31 2022] POST /identity/v3/projects => generated 375 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:13:31.661147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ebcdd-4372-4fc7-bd48-866967d46696 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:31.682522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354ebcdd-4372-4fc7-bd48-866967d46696 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:31.746876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8152a-1b72-4389-926c-d45cbadfeec7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:31.747565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8152a-1b72-4389-926c-d45cbadfeec7 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:31.747954 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8152a-1b72-4389-926c-d45cbadfeec7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:31.748241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a8152a-1b72-4389-926c-d45cbadfeec7 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:31.748954 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8152a-1b72-4389-926c-d45cbadfeec7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:31.751052 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a8152a-1b72-4389-926c-d45cbadfeec7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:31.782959 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1180/2366] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:13:31 2022] POST /identity/v3/users => generated 288 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:13:31.787274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f9f0f-8d35-4474-84aa-24878bf1df5e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:31.797514 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-791f9f0f-8d35-4474-84aa-24878bf1df5e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:31.802242 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaca6d3-a962-4ce6-85ae-693d63472922 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:31.802918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaca6d3-a962-4ce6-85ae-693d63472922 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:31.803211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaca6d3-a962-4ce6-85ae-693d63472922 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:31.803498 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaca6d3-a962-4ce6-85ae-693d63472922 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:31.804551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaca6d3-a962-4ce6-85ae-693d63472922 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:31.806211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaca6d3-a962-4ce6-85ae-693d63472922 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:31.816783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1187/2367] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:13:31.820929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8152a-1b72-4389-926c-d45cbadfeec7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:31.831328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a8152a-1b72-4389-926c-d45cbadfeec7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:31.836553 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939e2c39-5aac-4e8d-a876-0e4eb39dd165 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:31.836730 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e2c39-5aac-4e8d-a876-0e4eb39dd165 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:31.837023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e2c39-5aac-4e8d-a876-0e4eb39dd165 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:31.837393 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939e2c39-5aac-4e8d-a876-0e4eb39dd165 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:31.838135 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e2c39-5aac-4e8d-a876-0e4eb39dd165 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:31.839862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939e2c39-5aac-4e8d-a876-0e4eb39dd165 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:31.850125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1181/2368] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:13:31.854506 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaca6d3-a962-4ce6-85ae-693d63472922 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:31.865064 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaca6d3-a962-4ce6-85ae-693d63472922 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:31.870781 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03ed7f-1001-4a69-8ffd-9da5aa40063b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:31.871727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03ed7f-1001-4a69-8ffd-9da5aa40063b admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:31.872055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03ed7f-1001-4a69-8ffd-9da5aa40063b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:31.872341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c03ed7f-1001-4a69-8ffd-9da5aa40063b admin admin] PATH_INFO: `/v3/projects/04d28d327942493584a3fb9a00648fc4/users/93a7b1b712e2414d9310b09220076fda/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:31.888770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03ed7f-1001-4a69-8ffd-9da5aa40063b admin admin] RBAC: Authorizing `identity:create_grant(project_id=04d28d327942493584a3fb9a00648fc4, user_id=93a7b1b712e2414d9310b09220076fda, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:31.891501 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c03ed7f-1001-4a69-8ffd-9da5aa40063b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:31.907460 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1188/2369] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:13:31 2022] PUT /identity/v3/projects/04d28d327942493584a3fb9a00648fc4/users/93a7b1b712e2414d9310b09220076fda/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:31.918207 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859f15a-4e72-473c-ab05-34d41bccb343 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:31.918577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859f15a-4e72-473c-ab05-34d41bccb343 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:31.918934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a859f15a-4e72-473c-ab05-34d41bccb343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:31.945991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-a859f15a-4e72-473c-ab05-34d41bccb343 None None] MFA Rules not processed for user `93a7b1b712e2414d9310b09220076fda`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:31.947270 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a859f15a-4e72-473c-ab05-34d41bccb343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:32.011284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1182/2370] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:31 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:32.016050 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03ed7f-1001-4a69-8ffd-9da5aa40063b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:32.080994 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c03ed7f-1001-4a69-8ffd-9da5aa40063b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:32.086248 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29b57e-4c20-4011-a0a7-7670529562d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:32.086848 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29b57e-4c20-4011-a0a7-7670529562d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:32.087141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29b57e-4c20-4011-a0a7-7670529562d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:32.087421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a29b57e-4c20-4011-a0a7-7670529562d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:32.088147 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29b57e-4c20-4011-a0a7-7670529562d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:32.090070 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a29b57e-4c20-4011-a0a7-7670529562d3 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:32.118410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1189/2371] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:13:32 2022] POST /identity/v3/projects => generated 377 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:13:32.122900 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859f15a-4e72-473c-ab05-34d41bccb343 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:32.134911 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a859f15a-4e72-473c-ab05-34d41bccb343 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:32.200582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5913653-54b6-42c6-aee7-43831e004984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:32.201158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5913653-54b6-42c6-aee7-43831e004984 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:32.201453 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5913653-54b6-42c6-aee7-43831e004984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:32.201721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5913653-54b6-42c6-aee7-43831e004984 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:32.202416 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5913653-54b6-42c6-aee7-43831e004984 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:32.204397 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5913653-54b6-42c6-aee7-43831e004984 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:32.219844 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:32.226755 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1183/2372] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:13:32 2022] POST /identity/v3/users => generated 289 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:13:32.230856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29b57e-4c20-4011-a0a7-7670529562d3 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:32.241377 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a29b57e-4c20-4011-a0a7-7670529562d3 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:32.246528 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95addf-65a8-41f6-90b6-d3412b6ecdf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:32.247203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95addf-65a8-41f6-90b6-d3412b6ecdf5 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:32.247485 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95addf-65a8-41f6-90b6-d3412b6ecdf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:32.247766 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b95addf-65a8-41f6-90b6-d3412b6ecdf5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:32.248623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95addf-65a8-41f6-90b6-d3412b6ecdf5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:32.250189 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b95addf-65a8-41f6-90b6-d3412b6ecdf5 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:32.261168 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1190/2373] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:13:32.265462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5913653-54b6-42c6-aee7-43831e004984 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:32.277198 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5913653-54b6-42c6-aee7-43831e004984 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:32.285885 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa5295c-7a7a-4216-aa7b-803bb51d9185 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:32.286778 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5295c-7a7a-4216-aa7b-803bb51d9185 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:32.287114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5295c-7a7a-4216-aa7b-803bb51d9185 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:32.287498 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa5295c-7a7a-4216-aa7b-803bb51d9185 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:32.288577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5295c-7a7a-4216-aa7b-803bb51d9185 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:32.291152 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa5295c-7a7a-4216-aa7b-803bb51d9185 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:32.305270 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1184/2374] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:13:32.307306 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95addf-65a8-41f6-90b6-d3412b6ecdf5 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:32.317816 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b95addf-65a8-41f6-90b6-d3412b6ecdf5 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:32.323704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a6d3d-0353-4217-9c5d-d7fa964b32d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:32.324706 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a6d3d-0353-4217-9c5d-d7fa964b32d0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:32.324997 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a6d3d-0353-4217-9c5d-d7fa964b32d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:32.325281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87a6d3d-0353-4217-9c5d-d7fa964b32d0 admin admin] PATH_INFO: `/v3/projects/1a337a37f0ab44efb143c1666d9a2536/users/3eb44632195c43178b30b0d247930ba2/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:32.342349 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a6d3d-0353-4217-9c5d-d7fa964b32d0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1a337a37f0ab44efb143c1666d9a2536, user_id=3eb44632195c43178b30b0d247930ba2, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:32.345090 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87a6d3d-0353-4217-9c5d-d7fa964b32d0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:32.361291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1191/2375] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:13:32 2022] PUT /identity/v3/projects/1a337a37f0ab44efb143c1666d9a2536/users/3eb44632195c43178b30b0d247930ba2/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:13:32.373265 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6d9a6-5437-4d23-8b98-a4f6aeb9aa6a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:32.373565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6d9a6-5437-4d23-8b98-a4f6aeb9aa6a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:32.373852 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0db6d9a6-5437-4d23-8b98-a4f6aeb9aa6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:32.402560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0db6d9a6-5437-4d23-8b98-a4f6aeb9aa6a None None] MFA Rules not processed for user `3eb44632195c43178b30b0d247930ba2`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:32.403851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0db6d9a6-5437-4d23-8b98-a4f6aeb9aa6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:32.467990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1185/2376] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:32 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:32.472855 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a6d3d-0353-4217-9c5d-d7fa964b32d0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:32.492934 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:32.541672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87a6d3d-0353-4217-9c5d-d7fa964b32d0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:32.547610 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beecc8ee-03a5-4c19-beb9-5f75732782ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:32.548434 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecc8ee-03a5-4c19-beb9-5f75732782ce admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:32.548724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecc8ee-03a5-4c19-beb9-5f75732782ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:32.549027 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beecc8ee-03a5-4c19-beb9-5f75732782ce admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:32.549713 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beecc8ee-03a5-4c19-beb9-5f75732782ce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:32.551699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beecc8ee-03a5-4c19-beb9-5f75732782ce admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:32.583427 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1192/2377] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:13:32 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:13:32.588533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6d9a6-5437-4d23-8b98-a4f6aeb9aa6a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:32.602811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0db6d9a6-5437-4d23-8b98-a4f6aeb9aa6a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:32.661199 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37199b6-64f3-4c31-9bf4-4a01bdb3524a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:32.661868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37199b6-64f3-4c31-9bf4-4a01bdb3524a admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:32.662161 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37199b6-64f3-4c31-9bf4-4a01bdb3524a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:32.662442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37199b6-64f3-4c31-9bf4-4a01bdb3524a admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:32.663136 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37199b6-64f3-4c31-9bf4-4a01bdb3524a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:32.665111 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b37199b6-64f3-4c31-9bf4-4a01bdb3524a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:32.687048 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1186/2378] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:13:32 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:13:32.690993 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beecc8ee-03a5-4c19-beb9-5f75732782ce admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:32.701727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beecc8ee-03a5-4c19-beb9-5f75732782ce admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:32.706911 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e78919-9a13-4d8a-9cc9-3841bb5e5a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:32.707588 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e78919-9a13-4d8a-9cc9-3841bb5e5a0c admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:32.707938 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e78919-9a13-4d8a-9cc9-3841bb5e5a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:32.708223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e78919-9a13-4d8a-9cc9-3841bb5e5a0c admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:32.709007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e78919-9a13-4d8a-9cc9-3841bb5e5a0c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:32.710741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e78919-9a13-4d8a-9cc9-3841bb5e5a0c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:32.721995 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1193/2379] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:13:32.726186 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37199b6-64f3-4c31-9bf4-4a01bdb3524a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:32.736976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37199b6-64f3-4c31-9bf4-4a01bdb3524a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:32.742192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf19460-760b-47ce-ae10-81bece70ba67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:32.743184 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf19460-760b-47ce-ae10-81bece70ba67 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:32.743498 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf19460-760b-47ce-ae10-81bece70ba67 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:32.743803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf19460-760b-47ce-ae10-81bece70ba67 admin admin] PATH_INFO: `/v3/projects/1f2845617a834e02b0b41fbc5da45297/users/f741db27d0d64c348bddd0b7c7402111/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:32.763342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf19460-760b-47ce-ae10-81bece70ba67 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f2845617a834e02b0b41fbc5da45297, user_id=f741db27d0d64c348bddd0b7c7402111, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:32.766079 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcf19460-760b-47ce-ae10-81bece70ba67 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:32.780177 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1187/2380] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:13:32 2022] PUT /identity/v3/projects/1f2845617a834e02b0b41fbc5da45297/users/f741db27d0d64c348bddd0b7c7402111/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:32.784323 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e78919-9a13-4d8a-9cc9-3841bb5e5a0c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:32.794639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e78919-9a13-4d8a-9cc9-3841bb5e5a0c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:32.852900 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67b64c4-e647-4bca-9832-fd907a508f33 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:32.853626 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b64c4-e647-4bca-9832-fd907a508f33 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:32.854026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b64c4-e647-4bca-9832-fd907a508f33 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:32.854313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67b64c4-e647-4bca-9832-fd907a508f33 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:32.855058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67b64c4-e647-4bca-9832-fd907a508f33 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:32.856710 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67b64c4-e647-4bca-9832-fd907a508f33 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:32.871408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1194/2381] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:32 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:13:32.875801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf19460-760b-47ce-ae10-81bece70ba67 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:32.887282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf19460-760b-47ce-ae10-81bece70ba67 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:32.891784 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbc3cc-3940-41bb-bc55-d8206414b782 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:32.892697 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbc3cc-3940-41bb-bc55-d8206414b782 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:32.892988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbc3cc-3940-41bb-bc55-d8206414b782 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:32.893263 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7edbc3cc-3940-41bb-bc55-d8206414b782 admin admin] PATH_INFO: `/v3/projects/1f2845617a834e02b0b41fbc5da45297/users/f741db27d0d64c348bddd0b7c7402111/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:32.895501 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbc3cc-3940-41bb-bc55-d8206414b782 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1f2845617a834e02b0b41fbc5da45297, user_id=f741db27d0d64c348bddd0b7c7402111, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:32.898211 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7edbc3cc-3940-41bb-bc55-d8206414b782 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:32.909072 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1188/2382] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:13:32 2022] PUT /identity/v3/projects/1f2845617a834e02b0b41fbc5da45297/users/f741db27d0d64c348bddd0b7c7402111/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:32.919408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62875b26-13f0-4cc3-9664-52d4fb18de83 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:32.919731 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62875b26-13f0-4cc3-9664-52d4fb18de83 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:32.920089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62875b26-13f0-4cc3-9664-52d4fb18de83 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:32.947785 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-62875b26-13f0-4cc3-9664-52d4fb18de83 None None] MFA Rules not processed for user `f741db27d0d64c348bddd0b7c7402111`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:32.949079 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-62875b26-13f0-4cc3-9664-52d4fb18de83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:33.021250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1195/2383] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:32 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:33.032652 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbc3cc-3940-41bb-bc55-d8206414b782 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:33.034735 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7edbc3cc-3940-41bb-bc55-d8206414b782 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:33.067914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:33.096478 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7edbc3cc-3940-41bb-bc55-d8206414b782 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:33.100710 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935628e2-6ab9-49ea-8db9-1ef78067fedb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:33.101387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935628e2-6ab9-49ea-8db9-1ef78067fedb service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:33.101664 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935628e2-6ab9-49ea-8db9-1ef78067fedb service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:33.101943 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935628e2-6ab9-49ea-8db9-1ef78067fedb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:33.158474 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935628e2-6ab9-49ea-8db9-1ef78067fedb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:33.159961 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935628e2-6ab9-49ea-8db9-1ef78067fedb service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:33.170791 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1189/2384] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:33 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:34.672085 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62875b26-13f0-4cc3-9664-52d4fb18de83 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:34.673752 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-62875b26-13f0-4cc3-9664-52d4fb18de83 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:34.686445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62875b26-13f0-4cc3-9664-52d4fb18de83 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:34.690428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270569d-a34d-44dc-83ae-99860facedbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:34.691185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270569d-a34d-44dc-83ae-99860facedbd service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:34.692218 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270569d-a34d-44dc-83ae-99860facedbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:34.692571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9270569d-a34d-44dc-83ae-99860facedbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:34.702907 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:34.751204 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270569d-a34d-44dc-83ae-99860facedbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:34.752735 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9270569d-a34d-44dc-83ae-99860facedbd service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:34.762147 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1196/2385] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:34 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:35.325111 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935628e2-6ab9-49ea-8db9-1ef78067fedb service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:35.334830 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935628e2-6ab9-49ea-8db9-1ef78067fedb service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:35.339431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a4e775-59b0-4538-9b6c-1d5d2b359be7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:35.339995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4e775-59b0-4538-9b6c-1d5d2b359be7 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:35.340363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4e775-59b0-4538-9b6c-1d5d2b359be7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:35.340717 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a4e775-59b0-4538-9b6c-1d5d2b359be7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:35.352674 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4e775-59b0-4538-9b6c-1d5d2b359be7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:35.354347 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5a4e775-59b0-4538-9b6c-1d5d2b359be7 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:35.365107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1190/2386] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:35 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:39.273956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270569d-a34d-44dc-83ae-99860facedbd service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:39.346722 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9270569d-a34d-44dc-83ae-99860facedbd service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:39.351882 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23453e7-e43b-434a-9f2c-490dbfb1cefe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:39.353406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23453e7-e43b-434a-9f2c-490dbfb1cefe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:39.353705 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23453e7-e43b-434a-9f2c-490dbfb1cefe admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:39.353991 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f23453e7-e43b-434a-9f2c-490dbfb1cefe admin admin] PATH_INFO: `/v3/users/93a7b1b712e2414d9310b09220076fda` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:39.355284 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23453e7-e43b-434a-9f2c-490dbfb1cefe admin admin] RBAC: Authorizing `identity:delete_user(user_id=93a7b1b712e2414d9310b09220076fda)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:39.357489 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f23453e7-e43b-434a-9f2c-490dbfb1cefe admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:39.406608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f23453e7-e43b-434a-9f2c-490dbfb1cefe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '93a7b1b712e2414d9310b09220076fda'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:39.407344 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f23453e7-e43b-434a-9f2c-490dbfb1cefe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '93a7b1b712e2414d9310b09220076fda'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:39.419059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f23453e7-e43b-434a-9f2c-490dbfb1cefe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '93a7b1b712e2414d9310b09220076fda'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:39.427455 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f23453e7-e43b-434a-9f2c-490dbfb1cefe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '93a7b1b712e2414d9310b09220076fda'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:39.428153 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f23453e7-e43b-434a-9f2c-490dbfb1cefe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '93a7b1b712e2414d9310b09220076fda'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:39.441610 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1197/2387] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:13:39 2022] DELETE /identity/v3/users/93a7b1b712e2414d9310b09220076fda => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:39.446170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4eb5e-8adb-4dcd-8a45-82f5de73336b None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:39.446704 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4eb5e-8adb-4dcd-8a45-82f5de73336b None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:39.446898 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d4eb5e-8adb-4dcd-8a45-82f5de73336b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:39.464884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-45d4eb5e-8adb-4dcd-8a45-82f5de73336b None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:39.466039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-45d4eb5e-8adb-4dcd-8a45-82f5de73336b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:39.527345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1191/2388] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:39.536981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23453e7-e43b-434a-9f2c-490dbfb1cefe admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:39.538516 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f23453e7-e43b-434a-9f2c-490dbfb1cefe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:39.573327 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:39.600842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f23453e7-e43b-434a-9f2c-490dbfb1cefe admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:39.604774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8cdafa-6280-48c2-9d5b-bdf2e71c91a3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:39.605436 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cdafa-6280-48c2-9d5b-bdf2e71c91a3 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:39.605729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cdafa-6280-48c2-9d5b-bdf2e71c91a3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:39.606016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8cdafa-6280-48c2-9d5b-bdf2e71c91a3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:39.615545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8cdafa-6280-48c2-9d5b-bdf2e71c91a3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:39.617041 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8cdafa-6280-48c2-9d5b-bdf2e71c91a3 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:39.627213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1198/2389] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:39.906089 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4eb5e-8adb-4dcd-8a45-82f5de73336b None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:39.907911 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-45d4eb5e-8adb-4dcd-8a45-82f5de73336b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:39.923551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d4eb5e-8adb-4dcd-8a45-82f5de73336b None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:39.929018 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc45862-ca3f-4457-b556-6f42104c4381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:39.930009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc45862-ca3f-4457-b556-6f42104c4381 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:39.930360 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc45862-ca3f-4457-b556-6f42104c4381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:39.930708 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc45862-ca3f-4457-b556-6f42104c4381 admin admin] PATH_INFO: `/v3/projects/04d28d327942493584a3fb9a00648fc4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:39.932173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc45862-ca3f-4457-b556-6f42104c4381 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04d28d327942493584a3fb9a00648fc4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:39.934463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc45862-ca3f-4457-b556-6f42104c4381 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:39.994698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8bc45862-ca3f-4457-b556-6f42104c4381 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04d28d327942493584a3fb9a00648fc4', 'request_id': 'req-8bc45862-ca3f-4457-b556-6f42104c4381'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:40.004167 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8bc45862-ca3f-4457-b556-6f42104c4381 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04d28d327942493584a3fb9a00648fc4', 'request_id': 'req-8bc45862-ca3f-4457-b556-6f42104c4381'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:40.007843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601becc-8661-4052-86ff-e4337a2f6685 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:40.008335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601becc-8661-4052-86ff-e4337a2f6685 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:40.008623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f601becc-8661-4052-86ff-e4337a2f6685 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:40.012127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8bc45862-ca3f-4457-b556-6f42104c4381 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04d28d327942493584a3fb9a00648fc4', 'request_id': 'req-8bc45862-ca3f-4457-b556-6f42104c4381'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:40.016877 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:40.028326 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f601becc-8661-4052-86ff-e4337a2f6685 None None] MFA Rules not processed for user `066de1f11f724eee9df0377beb58d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:40.030437 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f601becc-8661-4052-86ff-e4337a2f6685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:40.031441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8bc45862-ca3f-4457-b556-6f42104c4381 admin admin] The token cache is being invalidate because project 04d28d327942493584a3fb9a00648fc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:13:40.032009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8bc45862-ca3f-4457-b556-6f42104c4381 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:40.032916 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8bc45862-ca3f-4457-b556-6f42104c4381 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:40.036903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1192/2390] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:13:39 2022] DELETE /identity/v3/projects/04d28d327942493584a3fb9a00648fc4 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:40.040888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc45862-ca3f-4457-b556-6f42104c4381 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:40.042326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8bc45862-ca3f-4457-b556-6f42104c4381 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:40.096302 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1199/2391] 10.222.0.85 () {58 vars in 1009 bytes} [Wed May 25 16:13:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:13:40.112322 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc45862-ca3f-4457-b556-6f42104c4381 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:40.118050 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d421844-b197-4a26-be79-d0904295501e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:40.118952 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d421844-b197-4a26-be79-d0904295501e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:40.119277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d421844-b197-4a26-be79-d0904295501e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:40.119612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d421844-b197-4a26-be79-d0904295501e admin admin] PATH_INFO: `/v3/users/3eb44632195c43178b30b0d247930ba2` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:40.120875 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d421844-b197-4a26-be79-d0904295501e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3eb44632195c43178b30b0d247930ba2)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:40.122826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d421844-b197-4a26-be79-d0904295501e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:40.143175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601becc-8661-4052-86ff-e4337a2f6685 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:40.144704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f601becc-8661-4052-86ff-e4337a2f6685 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:40.180256 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3d421844-b197-4a26-be79-d0904295501e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3eb44632195c43178b30b0d247930ba2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:40.192849 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3d421844-b197-4a26-be79-d0904295501e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3eb44632195c43178b30b0d247930ba2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:40.200601 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3d421844-b197-4a26-be79-d0904295501e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3eb44632195c43178b30b0d247930ba2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:40.201245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3d421844-b197-4a26-be79-d0904295501e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3eb44632195c43178b30b0d247930ba2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:40.208888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3d421844-b197-4a26-be79-d0904295501e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3eb44632195c43178b30b0d247930ba2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:40.215147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1193/2392] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:13:40 2022] DELETE /identity/v3/users/3eb44632195c43178b30b0d247930ba2 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:40.220281 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f601becc-8661-4052-86ff-e4337a2f6685 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:40.280852 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847eaad-bc6e-4957-82db-220b0a7978cc service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77fda27641c84fccb68a5cdaf4e8dc4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:40.281582 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847eaad-bc6e-4957-82db-220b0a7978cc service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:40.281767 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847eaad-bc6e-4957-82db-220b0a7978cc service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:40.282075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8847eaad-bc6e-4957-82db-220b0a7978cc service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:40.284012 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8847eaad-bc6e-4957-82db-220b0a7978cc service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:40.318817 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:40.356937 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847eaad-bc6e-4957-82db-220b0a7978cc service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:40.358566 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8847eaad-bc6e-4957-82db-220b0a7978cc service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:40.369574 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1200/2393] 10.222.0.85 () {60 vars in 1384 bytes} [Wed May 25 16:13:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 16:13:40.756235 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d421844-b197-4a26-be79-d0904295501e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:40.757801 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3d421844-b197-4a26-be79-d0904295501e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:40.843318 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d421844-b197-4a26-be79-d0904295501e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:40.849008 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:40.849882 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:40.850172 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:40.850457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904 admin admin] PATH_INFO: `/v3/projects/1a337a37f0ab44efb143c1666d9a2536` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:40.851702 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1a337a37f0ab44efb143c1666d9a2536)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:40.853721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:40.879396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:40.915690 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1a337a37f0ab44efb143c1666d9a2536', 'request_id': 'req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:40.927755 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1a337a37f0ab44efb143c1666d9a2536', 'request_id': 'req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:40.935798 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1a337a37f0ab44efb143c1666d9a2536', 'request_id': 'req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:40.952957 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904 admin admin] The token cache is being invalidate because project 1a337a37f0ab44efb143c1666d9a2536 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:13:40.953297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:40.953929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:40.955702 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1194/2394] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:13:40 2022] DELETE /identity/v3/projects/1a337a37f0ab44efb143c1666d9a2536 => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:40.963728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847eaad-bc6e-4957-82db-220b0a7978cc service swift] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:40.987013 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8847eaad-bc6e-4957-82db-220b0a7978cc service swift] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:41.058130 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f5c93-fa50-4b4b-b0ba-96321d47d794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:41.059174 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f5c93-fa50-4b4b-b0ba-96321d47d794 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:41.059533 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f5c93-fa50-4b4b-b0ba-96321d47d794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:41.059920 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f5c93-fa50-4b4b-b0ba-96321d47d794 admin admin] PATH_INFO: `/v3/users/f741db27d0d64c348bddd0b7c7402111` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:41.061265 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f5c93-fa50-4b4b-b0ba-96321d47d794 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f741db27d0d64c348bddd0b7c7402111)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:41.063345 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f5c93-fa50-4b4b-b0ba-96321d47d794 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:41.129010 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-331f5c93-fa50-4b4b-b0ba-96321d47d794 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f741db27d0d64c348bddd0b7c7402111'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:41.129861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-331f5c93-fa50-4b4b-b0ba-96321d47d794 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f741db27d0d64c348bddd0b7c7402111'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:41.156928 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-331f5c93-fa50-4b4b-b0ba-96321d47d794 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f741db27d0d64c348bddd0b7c7402111'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:41.169588 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-331f5c93-fa50-4b4b-b0ba-96321d47d794 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f741db27d0d64c348bddd0b7c7402111'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:41.170792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-331f5c93-fa50-4b4b-b0ba-96321d47d794 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f741db27d0d64c348bddd0b7c7402111'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:41.191637 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1201/2395] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:13:40 2022] DELETE /identity/v3/users/f741db27d0d64c348bddd0b7c7402111 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:41.600365 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:41.602133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:41.625281 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f945d8-8f4d-4258-ae8c-1923dbdb1904 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:41.713523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f66de01-0ff2-4a10-92e9-2e79d6106336 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:41.714444 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66de01-0ff2-4a10-92e9-2e79d6106336 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:41.714734 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66de01-0ff2-4a10-92e9-2e79d6106336 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:41.715012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f66de01-0ff2-4a10-92e9-2e79d6106336 admin admin] PATH_INFO: `/v3/projects/1f2845617a834e02b0b41fbc5da45297` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:41.717070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66de01-0ff2-4a10-92e9-2e79d6106336 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1f2845617a834e02b0b41fbc5da45297)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:41.718319 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f66de01-0ff2-4a10-92e9-2e79d6106336 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:41.781642 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2f66de01-0ff2-4a10-92e9-2e79d6106336 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1f2845617a834e02b0b41fbc5da45297', 'request_id': 'req-2f66de01-0ff2-4a10-92e9-2e79d6106336'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:41.794626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2f66de01-0ff2-4a10-92e9-2e79d6106336 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1f2845617a834e02b0b41fbc5da45297', 'request_id': 'req-2f66de01-0ff2-4a10-92e9-2e79d6106336'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:41.808746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2f66de01-0ff2-4a10-92e9-2e79d6106336 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1f2845617a834e02b0b41fbc5da45297', 'request_id': 'req-2f66de01-0ff2-4a10-92e9-2e79d6106336'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:41.820968 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2f66de01-0ff2-4a10-92e9-2e79d6106336 admin admin] The token cache is being invalidate because project 1f2845617a834e02b0b41fbc5da45297 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:13:41.821420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2f66de01-0ff2-4a10-92e9-2e79d6106336 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:41.822254 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-2f66de01-0ff2-4a10-92e9-2e79d6106336 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:13:41.824358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1195/2396] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:13:41 2022] DELETE /identity/v3/projects/1f2845617a834e02b0b41fbc5da45297 => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:13:41.829588 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5707dc7-2308-487c-afef-bc69e6dc8a6b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:41.830450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5707dc7-2308-487c-afef-bc69e6dc8a6b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:41.830947 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5707dc7-2308-487c-afef-bc69e6dc8a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:41.852676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b5707dc7-2308-487c-afef-bc69e6dc8a6b None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:41.854046 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b5707dc7-2308-487c-afef-bc69e6dc8a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:41.899394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:41.925995 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1202/2397] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:41.935878 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a193e6-f8e9-41fd-adfd-966b360f4e85 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:41.935878 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a193e6-f8e9-41fd-adfd-966b360f4e85 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:41.935878 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a193e6-f8e9-41fd-adfd-966b360f4e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:41.957071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-91a193e6-f8e9-41fd-adfd-966b360f4e85 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:41.958367 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-91a193e6-f8e9-41fd-adfd-966b360f4e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:41.967286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1196/2398] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:13:41.971740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5707dc7-2308-487c-afef-bc69e6dc8a6b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:41.984579 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5707dc7-2308-487c-afef-bc69e6dc8a6b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:41.990555 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005cbb77-142c-4720-8d62-0d9990504306 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:41.991452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005cbb77-142c-4720-8d62-0d9990504306 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:41.991743 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005cbb77-142c-4720-8d62-0d9990504306 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:41.992068 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-005cbb77-142c-4720-8d62-0d9990504306 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:41.992867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005cbb77-142c-4720-8d62-0d9990504306 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:13:41.993194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005cbb77-142c-4720-8d62-0d9990504306 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:41.994932 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-005cbb77-142c-4720-8d62-0d9990504306 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:42.013334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1203/2399] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:13:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:13:42.014780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a193e6-f8e9-41fd-adfd-966b360f4e85 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:42.025417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a193e6-f8e9-41fd-adfd-966b360f4e85 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:42.032737 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27696726-5614-4d59-adf4-3121e38b9fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:42.033431 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27696726-5614-4d59-adf4-3121e38b9fbe admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:42.033733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27696726-5614-4d59-adf4-3121e38b9fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:42.034016 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27696726-5614-4d59-adf4-3121e38b9fbe admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:42.034889 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27696726-5614-4d59-adf4-3121e38b9fbe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:42.037026 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27696726-5614-4d59-adf4-3121e38b9fbe admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:42.071494 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1197/2400] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:13:42 2022] POST /identity/v3/projects => generated 373 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:13:42.073305 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005cbb77-142c-4720-8d62-0d9990504306 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:42.087522 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-005cbb77-142c-4720-8d62-0d9990504306 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:42.151222 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b56f5e-38e8-46ed-a076-2a9039d22be0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:42.152282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b56f5e-38e8-46ed-a076-2a9039d22be0 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:42.152580 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b56f5e-38e8-46ed-a076-2a9039d22be0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:42.152864 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b56f5e-38e8-46ed-a076-2a9039d22be0 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:42.153577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b56f5e-38e8-46ed-a076-2a9039d22be0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:42.155597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b56f5e-38e8-46ed-a076-2a9039d22be0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:42.184811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1204/2401] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:13:42 2022] POST /identity/v3/users => generated 287 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:13:42.188951 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27696726-5614-4d59-adf4-3121e38b9fbe admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:42.200425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:42.207377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27696726-5614-4d59-adf4-3121e38b9fbe admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:42.213512 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfdcc60-586b-4fcf-a868-c3a294778cd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:42.215785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdcc60-586b-4fcf-a868-c3a294778cd0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:42.216511 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdcc60-586b-4fcf-a868-c3a294778cd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:42.217280 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfdcc60-586b-4fcf-a868-c3a294778cd0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:42.218247 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfdcc60-586b-4fcf-a868-c3a294778cd0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:42.220006 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfdcc60-586b-4fcf-a868-c3a294778cd0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:42.235881 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1198/2402] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:13:42.237127 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b56f5e-38e8-46ed-a076-2a9039d22be0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:42.247885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b56f5e-38e8-46ed-a076-2a9039d22be0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:42.253346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f8d254-c8b5-4cfd-ba7d-c424b2bac7de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:42.254208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8d254-c8b5-4cfd-ba7d-c424b2bac7de admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:42.254608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8d254-c8b5-4cfd-ba7d-c424b2bac7de admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:42.254956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f8d254-c8b5-4cfd-ba7d-c424b2bac7de admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:42.255696 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8d254-c8b5-4cfd-ba7d-c424b2bac7de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:42.257370 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f8d254-c8b5-4cfd-ba7d-c424b2bac7de admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:42.269539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1205/2403] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:13:42 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:13:42.272648 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfdcc60-586b-4fcf-a868-c3a294778cd0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:42.283363 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfdcc60-586b-4fcf-a868-c3a294778cd0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:42.293076 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026b33b-28c2-446d-a757-668042404b27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:42.294004 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026b33b-28c2-446d-a757-668042404b27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:42.294287 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026b33b-28c2-446d-a757-668042404b27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:42.294587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026b33b-28c2-446d-a757-668042404b27 admin admin] PATH_INFO: `/v3/projects/234b2b74107748719889e8c695484040/users/40052f39d8ad48f3ab5af37806ebfcb3/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:42.316291 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026b33b-28c2-446d-a757-668042404b27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=234b2b74107748719889e8c695484040, user_id=40052f39d8ad48f3ab5af37806ebfcb3, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:42.319232 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3026b33b-28c2-446d-a757-668042404b27 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:42.336401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1199/2404] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:13:42 2022] PUT /identity/v3/projects/234b2b74107748719889e8c695484040/users/40052f39d8ad48f3ab5af37806ebfcb3/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:13:42.341331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa31d0-c2e9-4bdb-b18d-be1366a04ebf None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:42.341630 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa31d0-c2e9-4bdb-b18d-be1366a04ebf None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:42.341910 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfa31d0-c2e9-4bdb-b18d-be1366a04ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:42.360425 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6dfa31d0-c2e9-4bdb-b18d-be1366a04ebf None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:42.361628 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6dfa31d0-c2e9-4bdb-b18d-be1366a04ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:42.436261 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1206/2405] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:42.448645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026b33b-28c2-446d-a757-668042404b27 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:42.450788 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3026b33b-28c2-446d-a757-668042404b27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:42.525699 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026b33b-28c2-446d-a757-668042404b27 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:42.529953 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e038eef-39d1-4aaf-a487-01b5fb485e34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:42.531148 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e038eef-39d1-4aaf-a487-01b5fb485e34 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:42.531549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e038eef-39d1-4aaf-a487-01b5fb485e34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:42.531939 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e038eef-39d1-4aaf-a487-01b5fb485e34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:42.545767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e038eef-39d1-4aaf-a487-01b5fb485e34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:42.547265 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e038eef-39d1-4aaf-a487-01b5fb485e34 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:42.560557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1200/2406] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:43.342787 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebcb4f-3e62-4bda-be73-f1f7eaefc92c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:43.343318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebcb4f-3e62-4bda-be73-f1f7eaefc92c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:43.343418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3ebcb4f-3e62-4bda-be73-f1f7eaefc92c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:43.373724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d3ebcb4f-3e62-4bda-be73-f1f7eaefc92c None None] MFA Rules not processed for user `40052f39d8ad48f3ab5af37806ebfcb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:43.375377 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d3ebcb4f-3e62-4bda-be73-f1f7eaefc92c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:43.437007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:43.479408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1207/2407] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:43.491255 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e038eef-39d1-4aaf-a487-01b5fb485e34 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:43.493224 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4e038eef-39d1-4aaf-a487-01b5fb485e34 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:43.566708 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e038eef-39d1-4aaf-a487-01b5fb485e34 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:43.572226 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06b3a24-1bb1-435d-82fb-852a828e26f7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:43.572937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b3a24-1bb1-435d-82fb-852a828e26f7 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:43.573231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b3a24-1bb1-435d-82fb-852a828e26f7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:43.573524 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06b3a24-1bb1-435d-82fb-852a828e26f7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:43.594636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b3a24-1bb1-435d-82fb-852a828e26f7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:43.596524 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06b3a24-1bb1-435d-82fb-852a828e26f7 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:43.618537 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1201/2408] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:13:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:13:45.087008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38926daa-91c4-478e-b272-aa1850e26e2d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:45.087547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38926daa-91c4-478e-b272-aa1850e26e2d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:45.087547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38926daa-91c4-478e-b272-aa1850e26e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:45.112519 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-38926daa-91c4-478e-b272-aa1850e26e2d None None] MFA Rules not processed for user `40052f39d8ad48f3ab5af37806ebfcb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:45.113807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-38926daa-91c4-478e-b272-aa1850e26e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:45.121736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1208/2409] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:13:45 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:53.603166 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=65,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:55.104419 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=13 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:13:58.839646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a647267-e452-43af-967e-2d13f6e79d10 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:58.840195 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a647267-e452-43af-967e-2d13f6e79d10 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:58.840295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a647267-e452-43af-967e-2d13f6e79d10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:58.857514 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1a647267-e452-43af-967e-2d13f6e79d10 None None] MFA Rules not processed for user `066de1f11f724eee9df0377beb58d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:13:58.858700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1a647267-e452-43af-967e-2d13f6e79d10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:58.916353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1202/2410] 10.222.0.85 () {58 vars in 1009 bytes} [Wed May 25 16:13:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:13:58.928580 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38926daa-91c4-478e-b272-aa1850e26e2d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:13:58.930017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-38926daa-91c4-478e-b272-aa1850e26e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:13:59.003029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38926daa-91c4-478e-b272-aa1850e26e2d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:13:59.008310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2709145-0825-4a8a-b151-c7de75fd3683 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77fda27641c84fccb68a5cdaf4e8dc4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:13:59.008964 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2709145-0825-4a8a-b151-c7de75fd3683 service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:13:59.009255 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2709145-0825-4a8a-b151-c7de75fd3683 service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:13:59.009535 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2709145-0825-4a8a-b151-c7de75fd3683 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:13:59.017546 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2709145-0825-4a8a-b151-c7de75fd3683 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:13:59.019144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2709145-0825-4a8a-b151-c7de75fd3683 service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:13:59.027774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1209/2411] 10.222.0.85 () {60 vars in 1384 bytes} [Wed May 25 16:13:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:14:08.908907 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=25 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:14:09.024198 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=32 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:14:09.523600 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940fa2d-3b9c-43d8-a987-fa0a1b0d4edb None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:09.523936 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940fa2d-3b9c-43d8-a987-fa0a1b0d4edb None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:09.524226 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0940fa2d-3b9c-43d8-a987-fa0a1b0d4edb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:09.542406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0940fa2d-3b9c-43d8-a987-fa0a1b0d4edb None None] MFA Rules not processed for user `40052f39d8ad48f3ab5af37806ebfcb3`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:09.543621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0940fa2d-3b9c-43d8-a987-fa0a1b0d4edb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:09.550843 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1203/2412] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:14:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:14:15.817299 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a634a8-97a0-49b4-961d-f52118231379 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:15.817822 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a634a8-97a0-49b4-961d-f52118231379 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:15.817929 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a634a8-97a0-49b4-961d-f52118231379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:15.836845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-51a634a8-97a0-49b4-961d-f52118231379 None None] MFA Rules not processed for user `066de1f11f724eee9df0377beb58d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:15.837981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-51a634a8-97a0-49b4-961d-f52118231379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:15.845726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1210/2413] 10.222.0.85 () {58 vars in 967 bytes} [Wed May 25 16:14:15 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:14:15.861428 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940fa2d-3b9c-43d8-a987-fa0a1b0d4edb None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:15.876615 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0940fa2d-3b9c-43d8-a987-fa0a1b0d4edb None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:15.883310 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c12570-b7af-4288-b697-c444a8389bc7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77fda27641c84fccb68a5cdaf4e8dc4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:15.884013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c12570-b7af-4288-b697-c444a8389bc7 service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:15.884307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c12570-b7af-4288-b697-c444a8389bc7 service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:15.884591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c12570-b7af-4288-b697-c444a8389bc7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:15.894035 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c12570-b7af-4288-b697-c444a8389bc7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:15.895690 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c12570-b7af-4288-b697-c444a8389bc7 service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:15.904758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1204/2414] 10.222.0.85 () {60 vars in 1384 bytes} [Wed May 25 16:14:15 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:14:25.831177 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:14:25.901515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=9 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:14:28.351780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b8551-7001-491f-b7b1-c894c1a106ba None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:28.352296 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b8551-7001-491f-b7b1-c894c1a106ba None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:28.352395 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b8551-7001-491f-b7b1-c894c1a106ba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:28.371441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b51b8551-7001-491f-b7b1-c894c1a106ba None None] MFA Rules not processed for user `066de1f11f724eee9df0377beb58d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:28.372791 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b51b8551-7001-491f-b7b1-c894c1a106ba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:28.380496 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1211/2415] 10.222.0.85 () {58 vars in 1009 bytes} [Wed May 25 16:14:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:14:28.392937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c12570-b7af-4288-b697-c444a8389bc7 service swift] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:28.403857 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c12570-b7af-4288-b697-c444a8389bc7 service swift] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:28.410335 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db90761-61b8-4b47-a3ae-3237e8330b1f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77fda27641c84fccb68a5cdaf4e8dc4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:28.411125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db90761-61b8-4b47-a3ae-3237e8330b1f service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:28.411433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db90761-61b8-4b47-a3ae-3237e8330b1f service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:28.411758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db90761-61b8-4b47-a3ae-3237e8330b1f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:28.422158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db90761-61b8-4b47-a3ae-3237e8330b1f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:28.423839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db90761-61b8-4b47-a3ae-3237e8330b1f service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:28.432375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1205/2416] 10.222.0.85 () {60 vars in 1384 bytes} [Wed May 25 16:14:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:14:38.365779 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:14:38.428790 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=6 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:14:44.031038 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703e3ab-d71c-46b1-9d12-6ede91765020 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:44.031579 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703e3ab-d71c-46b1-9d12-6ede91765020 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:44.031681 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a703e3ab-d71c-46b1-9d12-6ede91765020 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:44.060536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-a703e3ab-d71c-46b1-9d12-6ede91765020 None None] MFA Rules not processed for user `066de1f11f724eee9df0377beb58d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:44.061853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a703e3ab-d71c-46b1-9d12-6ede91765020 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:44.070712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1212/2417] 10.222.0.85 () {58 vars in 967 bytes} [Wed May 25 16:14:44 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:14:44.086871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db90761-61b8-4b47-a3ae-3237e8330b1f service swift] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:44.098065 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db90761-61b8-4b47-a3ae-3237e8330b1f service swift] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:44.104663 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8227a-4d4a-47db-876c-573c263ed333 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77fda27641c84fccb68a5cdaf4e8dc4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:44.105316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8227a-4d4a-47db-876c-573c263ed333 service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:44.105611 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8227a-4d4a-47db-876c-573c263ed333 service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:44.105894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d8227a-4d4a-47db-876c-573c263ed333 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:44.114293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8227a-4d4a-47db-876c-573c263ed333 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:44.115925 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d8227a-4d4a-47db-876c-573c263ed333 service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:44.124287 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1206/2418] 10.222.0.85 () {60 vars in 1384 bytes} [Wed May 25 16:14:44 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:14:45.761343 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703e3ab-d71c-46b1-9d12-6ede91765020 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:45.774266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a703e3ab-d71c-46b1-9d12-6ede91765020 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:45.777916 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e894035-b038-448c-a928-633e018c87e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:45.778578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e894035-b038-448c-a928-633e018c87e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:45.778868 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e894035-b038-448c-a928-633e018c87e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:45.779154 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e894035-b038-448c-a928-633e018c87e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:45.781087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4e894035-b038-448c-a928-633e018c87e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:45.795146 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e894035-b038-448c-a928-633e018c87e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:45.796881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e894035-b038-448c-a928-633e018c87e9 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:45.814624 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1213/2419] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:14:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:14:46.103579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8227a-4d4a-47db-876c-573c263ed333 service swift] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:46.105757 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-66d8227a-4d4a-47db-876c-573c263ed333 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:46.179785 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e894035-b038-448c-a928-633e018c87e9 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:46.191493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d8227a-4d4a-47db-876c-573c263ed333 service swift] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:46.194369 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e894035-b038-448c-a928-633e018c87e9 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:46.199131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8663a9-229f-48f7-b07f-297305e1f8b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b738f1b8c64590955a640c461b4ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:46.199743 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b2e27bf-6982-4071-89d7-afdb62eb6477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:46.200141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8663a9-229f-48f7-b07f-297305e1f8b3 service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:46.200604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8663a9-229f-48f7-b07f-297305e1f8b3 service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:46.200736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e27bf-6982-4071-89d7-afdb62eb6477 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:46.200919 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a8663a9-229f-48f7-b07f-297305e1f8b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:46.201021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e27bf-6982-4071-89d7-afdb62eb6477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:46.201257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2e27bf-6982-4071-89d7-afdb62eb6477 admin admin] PATH_INFO: `/v3/users/40052f39d8ad48f3ab5af37806ebfcb3` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:46.202438 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e27bf-6982-4071-89d7-afdb62eb6477 admin admin] RBAC: Authorizing `identity:delete_user(user_id=40052f39d8ad48f3ab5af37806ebfcb3)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:46.202852 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1a8663a9-229f-48f7-b07f-297305e1f8b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:46.204545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b2e27bf-6982-4071-89d7-afdb62eb6477 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:46.216245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8663a9-229f-48f7-b07f-297305e1f8b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:46.217942 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a8663a9-229f-48f7-b07f-297305e1f8b3 service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:46.232773 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1207/2420] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:14:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:14:46.259057 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7b2e27bf-6982-4071-89d7-afdb62eb6477 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '40052f39d8ad48f3ab5af37806ebfcb3'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:46.260425 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7b2e27bf-6982-4071-89d7-afdb62eb6477 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '40052f39d8ad48f3ab5af37806ebfcb3'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:46.274620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7b2e27bf-6982-4071-89d7-afdb62eb6477 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '40052f39d8ad48f3ab5af37806ebfcb3'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:46.284314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7b2e27bf-6982-4071-89d7-afdb62eb6477 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '40052f39d8ad48f3ab5af37806ebfcb3'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:46.285791 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7b2e27bf-6982-4071-89d7-afdb62eb6477 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '40052f39d8ad48f3ab5af37806ebfcb3'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:46.304675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1214/2421] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:14:46 2022] DELETE /identity/v3/users/40052f39d8ad48f3ab5af37806ebfcb3 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:14:46.486403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8663a9-229f-48f7-b07f-297305e1f8b3 service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:46.503295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a8663a9-229f-48f7-b07f-297305e1f8b3 service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:46.562051 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea2fcab9-f786-4baf-bfd8-28171dda4d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:46.562934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fcab9-f786-4baf-bfd8-28171dda4d89 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:46.563222 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fcab9-f786-4baf-bfd8-28171dda4d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:46.563506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea2fcab9-f786-4baf-bfd8-28171dda4d89 admin admin] PATH_INFO: `/v3/projects/234b2b74107748719889e8c695484040` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:46.564738 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2fcab9-f786-4baf-bfd8-28171dda4d89 admin admin] RBAC: Authorizing `identity:delete_project(project_id=234b2b74107748719889e8c695484040)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:46.566723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea2fcab9-f786-4baf-bfd8-28171dda4d89 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:46.630944 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ea2fcab9-f786-4baf-bfd8-28171dda4d89 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '234b2b74107748719889e8c695484040', 'request_id': 'req-ea2fcab9-f786-4baf-bfd8-28171dda4d89'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:46.639059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ea2fcab9-f786-4baf-bfd8-28171dda4d89 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '234b2b74107748719889e8c695484040', 'request_id': 'req-ea2fcab9-f786-4baf-bfd8-28171dda4d89'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:46.646096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ea2fcab9-f786-4baf-bfd8-28171dda4d89 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '234b2b74107748719889e8c695484040', 'request_id': 'req-ea2fcab9-f786-4baf-bfd8-28171dda4d89'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:46.658608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ea2fcab9-f786-4baf-bfd8-28171dda4d89 admin admin] The token cache is being invalidate because project 234b2b74107748719889e8c695484040 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:14:46.658906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ea2fcab9-f786-4baf-bfd8-28171dda4d89 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:46.659589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ea2fcab9-f786-4baf-bfd8-28171dda4d89 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:46.661150 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1208/2422] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:14:46 2022] DELETE /identity/v3/projects/234b2b74107748719889e8c695484040 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:14:46.666313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6739d-9bd5-4917-ae2f-9d58c1dab1e0 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:46.666599 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6739d-9bd5-4917-ae2f-9d58c1dab1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:46.666880 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c6739d-9bd5-4917-ae2f-9d58c1dab1e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:46.685366 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e2c6739d-9bd5-4917-ae2f-9d58c1dab1e0 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:46.686590 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e2c6739d-9bd5-4917-ae2f-9d58c1dab1e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:46.754468 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1215/2423] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:14:46.763875 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc4663-b17e-4b83-87a9-8287baa096b5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:46.764179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc4663-b17e-4b83-87a9-8287baa096b5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:46.764643 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdc4663-b17e-4b83-87a9-8287baa096b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:46.784695 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:14:46.792415 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-0bdc4663-b17e-4b83-87a9-8287baa096b5 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:46.795780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0bdc4663-b17e-4b83-87a9-8287baa096b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:46.807713 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1209/2424] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:14:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:14:46.812345 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6739d-9bd5-4917-ae2f-9d58c1dab1e0 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:46.823200 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c6739d-9bd5-4917-ae2f-9d58c1dab1e0 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:46.845307 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2848a-7bc5-493d-a5bd-c4df26978e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:46.846375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2848a-7bc5-493d-a5bd-c4df26978e4b admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:46.846702 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2848a-7bc5-493d-a5bd-c4df26978e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:46.846988 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb2848a-7bc5-493d-a5bd-c4df26978e4b admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:46.847994 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2848a-7bc5-493d-a5bd-c4df26978e4b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:14:46.848508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2848a-7bc5-493d-a5bd-c4df26978e4b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:46.850412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb2848a-7bc5-493d-a5bd-c4df26978e4b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:46.865914 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1216/2425] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:14:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:14:46.870141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc4663-b17e-4b83-87a9-8287baa096b5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:46.882156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdc4663-b17e-4b83-87a9-8287baa096b5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:46.891946 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3722ebc-a757-4082-88ca-95d745dc6d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:46.893292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3722ebc-a757-4082-88ca-95d745dc6d0e admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:46.893655 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3722ebc-a757-4082-88ca-95d745dc6d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:46.894004 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3722ebc-a757-4082-88ca-95d745dc6d0e admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:46.895651 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3722ebc-a757-4082-88ca-95d745dc6d0e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:46.897969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3722ebc-a757-4082-88ca-95d745dc6d0e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:46.951590 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1210/2426] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:14:46 2022] POST /identity/v3/projects => generated 371 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:14:46.955762 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2848a-7bc5-493d-a5bd-c4df26978e4b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:46.966150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb2848a-7bc5-493d-a5bd-c4df26978e4b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:47.008110 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:14:47.030671 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce1216-353d-4be6-a371-9c4f0d9e9886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:47.031416 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce1216-353d-4be6-a371-9c4f0d9e9886 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:47.031796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce1216-353d-4be6-a371-9c4f0d9e9886 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:47.032122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ce1216-353d-4be6-a371-9c4f0d9e9886 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:47.032845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce1216-353d-4be6-a371-9c4f0d9e9886 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:47.035121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ce1216-353d-4be6-a371-9c4f0d9e9886 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:47.060950 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1217/2427] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:14:46 2022] POST /identity/v3/users => generated 286 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:14:47.065233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3722ebc-a757-4082-88ca-95d745dc6d0e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:47.076129 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3722ebc-a757-4082-88ca-95d745dc6d0e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:47.080914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850bfb7c-acde-4418-aa7c-02afedda14ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:47.081591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bfb7c-acde-4418-aa7c-02afedda14ac admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:47.081876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bfb7c-acde-4418-aa7c-02afedda14ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:47.082154 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850bfb7c-acde-4418-aa7c-02afedda14ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:47.082883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850bfb7c-acde-4418-aa7c-02afedda14ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:47.084521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850bfb7c-acde-4418-aa7c-02afedda14ac admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:47.095466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1211/2428] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:14:47.100483 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce1216-353d-4be6-a371-9c4f0d9e9886 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:47.112469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ce1216-353d-4be6-a371-9c4f0d9e9886 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:47.121632 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60784635-598e-48c1-b451-c6122fa62574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:47.122741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60784635-598e-48c1-b451-c6122fa62574 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:47.123123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60784635-598e-48c1-b451-c6122fa62574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:47.123412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60784635-598e-48c1-b451-c6122fa62574 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:47.124618 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60784635-598e-48c1-b451-c6122fa62574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:47.126423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60784635-598e-48c1-b451-c6122fa62574 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:47.140086 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1218/2429] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:14:47 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:14:47.144005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850bfb7c-acde-4418-aa7c-02afedda14ac admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:47.154243 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850bfb7c-acde-4418-aa7c-02afedda14ac admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:47.159018 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7c255-b1c0-467d-91e9-66e3a3ed70e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:47.159959 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7c255-b1c0-467d-91e9-66e3a3ed70e6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:47.160248 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7c255-b1c0-467d-91e9-66e3a3ed70e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:47.160534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c7c255-b1c0-467d-91e9-66e3a3ed70e6 admin admin] PATH_INFO: `/v3/projects/7528adefacfc41c5a54869ec5603bde8/users/abd62b4b16da4dcd8e0bdbcb6d9db3a8/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:47.178093 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c7c255-b1c0-467d-91e9-66e3a3ed70e6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7528adefacfc41c5a54869ec5603bde8, user_id=abd62b4b16da4dcd8e0bdbcb6d9db3a8, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:47.181345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c7c255-b1c0-467d-91e9-66e3a3ed70e6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:47.198835 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1212/2430] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:14:47 2022] PUT /identity/v3/projects/7528adefacfc41c5a54869ec5603bde8/users/abd62b4b16da4dcd8e0bdbcb6d9db3a8/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:14:47.204227 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed00e1-b258-4aa4-ac2d-9e61f9858603 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:47.204407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed00e1-b258-4aa4-ac2d-9e61f9858603 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:47.204719 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ed00e1-b258-4aa4-ac2d-9e61f9858603 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:47.226944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-68ed00e1-b258-4aa4-ac2d-9e61f9858603 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:47.226944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-68ed00e1-b258-4aa4-ac2d-9e61f9858603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:47.334239 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1219/2431] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:14:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:14:47.343784 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7c255-b1c0-467d-91e9-66e3a3ed70e6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:47.346046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c7c7c255-b1c0-467d-91e9-66e3a3ed70e6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:47.411587 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c7c255-b1c0-467d-91e9-66e3a3ed70e6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:47.415241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0164fe5-c248-4a8d-ae09-9b7b6f09d32f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:47.415987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0164fe5-c248-4a8d-ae09-9b7b6f09d32f service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:47.416277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0164fe5-c248-4a8d-ae09-9b7b6f09d32f service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:47.416562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0164fe5-c248-4a8d-ae09-9b7b6f09d32f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:47.426737 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0164fe5-c248-4a8d-ae09-9b7b6f09d32f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:47.428648 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0164fe5-c248-4a8d-ae09-9b7b6f09d32f service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:47.439507 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1213/2432] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:14:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:14:49.808852 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed00e1-b258-4aa4-ac2d-9e61f9858603 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:49.810435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-68ed00e1-b258-4aa4-ac2d-9e61f9858603 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:49.914582 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0164fe5-c248-4a8d-ae09-9b7b6f09d32f service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:49.916457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e0164fe5-c248-4a8d-ae09-9b7b6f09d32f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:49.932098 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0164fe5-c248-4a8d-ae09-9b7b6f09d32f service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:49.932335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ed00e1-b258-4aa4-ac2d-9e61f9858603 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:49.938425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54b97592-173e-469b-b854-06d65966b2b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:49.939661 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b97592-173e-469b-b854-06d65966b2b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:49.940203 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b97592-173e-469b-b854-06d65966b2b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:49.940747 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b97592-173e-469b-b854-06d65966b2b8 admin admin] PATH_INFO: `/v3/users/708fdc537e1747d7bd5917ed7af9c368` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:49.942552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505eeb7e-2d4d-41dc-87f7-d8b5f45bcece service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b738f1b8c64590955a640c461b4ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:49.942999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505eeb7e-2d4d-41dc-87f7-d8b5f45bcece service placement] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:49.943302 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505eeb7e-2d4d-41dc-87f7-d8b5f45bcece service placement] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:49.943584 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505eeb7e-2d4d-41dc-87f7-d8b5f45bcece service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:49.945719 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-505eeb7e-2d4d-41dc-87f7-d8b5f45bcece service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:49.951798 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b97592-173e-469b-b854-06d65966b2b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=708fdc537e1747d7bd5917ed7af9c368)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:49.954261 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54b97592-173e-469b-b854-06d65966b2b8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:49.964623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505eeb7e-2d4d-41dc-87f7-d8b5f45bcece service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:49.966424 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-505eeb7e-2d4d-41dc-87f7-d8b5f45bcece service placement] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:49.996450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1220/2433] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:14:49 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:14:50.049648 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54b97592-173e-469b-b854-06d65966b2b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '708fdc537e1747d7bd5917ed7af9c368'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:50.063993 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54b97592-173e-469b-b854-06d65966b2b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '708fdc537e1747d7bd5917ed7af9c368'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:50.096380 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54b97592-173e-469b-b854-06d65966b2b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '708fdc537e1747d7bd5917ed7af9c368'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:50.098061 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54b97592-173e-469b-b854-06d65966b2b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '708fdc537e1747d7bd5917ed7af9c368'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:50.125258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-54b97592-173e-469b-b854-06d65966b2b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '708fdc537e1747d7bd5917ed7af9c368'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:50.138912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1214/2434] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:14:49 2022] DELETE /identity/v3/users/708fdc537e1747d7bd5917ed7af9c368 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:14:50.345962 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505eeb7e-2d4d-41dc-87f7-d8b5f45bcece service placement] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:50.361274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505eeb7e-2d4d-41dc-87f7-d8b5f45bcece service placement] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:50.421335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:14:50.435256 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e3666a-557e-42e9-94c7-29b3b70c14eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:50.436487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3666a-557e-42e9-94c7-29b3b70c14eb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:50.436609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3666a-557e-42e9-94c7-29b3b70c14eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:50.436810 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e3666a-557e-42e9-94c7-29b3b70c14eb admin admin] PATH_INFO: `/v3/projects/32e48e8e884b4840ba54d546d82d2334` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:50.438354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3666a-557e-42e9-94c7-29b3b70c14eb admin admin] RBAC: Authorizing `identity:delete_project(project_id=32e48e8e884b4840ba54d546d82d2334)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:50.440123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e3666a-557e-42e9-94c7-29b3b70c14eb admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:50.513920 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d4e3666a-557e-42e9-94c7-29b3b70c14eb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32e48e8e884b4840ba54d546d82d2334', 'request_id': 'req-d4e3666a-557e-42e9-94c7-29b3b70c14eb'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:50.524569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d4e3666a-557e-42e9-94c7-29b3b70c14eb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32e48e8e884b4840ba54d546d82d2334', 'request_id': 'req-d4e3666a-557e-42e9-94c7-29b3b70c14eb'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:50.536220 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d4e3666a-557e-42e9-94c7-29b3b70c14eb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32e48e8e884b4840ba54d546d82d2334', 'request_id': 'req-d4e3666a-557e-42e9-94c7-29b3b70c14eb'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:50.550334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d4e3666a-557e-42e9-94c7-29b3b70c14eb admin admin] The token cache is being invalidate because project 32e48e8e884b4840ba54d546d82d2334 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:14:50.550632 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d4e3666a-557e-42e9-94c7-29b3b70c14eb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:50.551377 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d4e3666a-557e-42e9-94c7-29b3b70c14eb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:14:50.553284 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1221/2435] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:14:50 2022] DELETE /identity/v3/projects/32e48e8e884b4840ba54d546d82d2334 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:14:50.558420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0d8a8-ed36-418d-a73a-b17608fa13dd None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:50.558719 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0d8a8-ed36-418d-a73a-b17608fa13dd None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:50.559012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a0d8a8-ed36-418d-a73a-b17608fa13dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:50.582353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-b3a0d8a8-ed36-418d-a73a-b17608fa13dd None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:50.583629 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b3a0d8a8-ed36-418d-a73a-b17608fa13dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:50.588340 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:14:50.693547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69973185-a3fc-49c3-8de6-f1618ec3d7ac None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:50.693955 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1215/2436] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:14:50.694089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69973185-a3fc-49c3-8de6-f1618ec3d7ac None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:50.694435 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69973185-a3fc-49c3-8de6-f1618ec3d7ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:50.725955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-69973185-a3fc-49c3-8de6-f1618ec3d7ac None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:50.727277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-69973185-a3fc-49c3-8de6-f1618ec3d7ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:50.747106 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1222/2437] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:14:50.754366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0d8a8-ed36-418d-a73a-b17608fa13dd None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:50.766580 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a0d8a8-ed36-418d-a73a-b17608fa13dd None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:50.771442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02c49f-1c96-4bdd-ba84-fd5392e52b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:50.772175 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02c49f-1c96-4bdd-ba84-fd5392e52b09 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:50.772469 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02c49f-1c96-4bdd-ba84-fd5392e52b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:50.772754 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b02c49f-1c96-4bdd-ba84-fd5392e52b09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:50.773496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02c49f-1c96-4bdd-ba84-fd5392e52b09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:14:50.773815 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02c49f-1c96-4bdd-ba84-fd5392e52b09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:50.775682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b02c49f-1c96-4bdd-ba84-fd5392e52b09 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:50.793909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1216/2438] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:14:50.794458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69973185-a3fc-49c3-8de6-f1618ec3d7ac None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:50.806313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69973185-a3fc-49c3-8de6-f1618ec3d7ac None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:50.818107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a98ba-c83c-44f9-bda9-cc7397514eff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:50.822630 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a98ba-c83c-44f9-bda9-cc7397514eff admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:50.823218 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a98ba-c83c-44f9-bda9-cc7397514eff admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:50.823672 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611a98ba-c83c-44f9-bda9-cc7397514eff admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:50.826058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a98ba-c83c-44f9-bda9-cc7397514eff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:50.828145 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-611a98ba-c83c-44f9-bda9-cc7397514eff admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:50.864291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1223/2439] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:14:50 2022] POST /identity/v3/projects => generated 365 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:14:50.868450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02c49f-1c96-4bdd-ba84-fd5392e52b09 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:50.884618 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b02c49f-1c96-4bdd-ba84-fd5392e52b09 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:50.959814 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef474ac-8e39-4c6f-9a66-72a6c043c8b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:50.960705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef474ac-8e39-4c6f-9a66-72a6c043c8b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:50.961256 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef474ac-8e39-4c6f-9a66-72a6c043c8b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:50.962508 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef474ac-8e39-4c6f-9a66-72a6c043c8b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:50.963440 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef474ac-8e39-4c6f-9a66-72a6c043c8b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:50.965654 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef474ac-8e39-4c6f-9a66-72a6c043c8b7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:51.002131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1217/2440] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:14:50 2022] POST /identity/v3/users => generated 283 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:14:51.003295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a98ba-c83c-44f9-bda9-cc7397514eff admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:51.015157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611a98ba-c83c-44f9-bda9-cc7397514eff admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:51.027352 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54d056-1047-4ab1-96b6-b0878b34af78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:51.029745 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54d056-1047-4ab1-96b6-b0878b34af78 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:51.030324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54d056-1047-4ab1-96b6-b0878b34af78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:51.031523 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b54d056-1047-4ab1-96b6-b0878b34af78 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:51.032896 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54d056-1047-4ab1-96b6-b0878b34af78 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:51.036474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b54d056-1047-4ab1-96b6-b0878b34af78 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:51.057197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef474ac-8e39-4c6f-9a66-72a6c043c8b7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:51.066995 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1224/2441] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:14:51.069849 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef474ac-8e39-4c6f-9a66-72a6c043c8b7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:51.075076 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c23432b-f1ae-4b37-8309-129e17186741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:51.075969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23432b-f1ae-4b37-8309-129e17186741 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:51.076728 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23432b-f1ae-4b37-8309-129e17186741 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:51.077234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c23432b-f1ae-4b37-8309-129e17186741 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:51.078644 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c23432b-f1ae-4b37-8309-129e17186741 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:51.080755 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c23432b-f1ae-4b37-8309-129e17186741 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:51.096494 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54d056-1047-4ab1-96b6-b0878b34af78 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:51.097585 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1218/2442] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:14:51 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:14:51.107874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b54d056-1047-4ab1-96b6-b0878b34af78 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:51.113546 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dee841-5318-4be4-9e8b-2af5baa524fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:51.114481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dee841-5318-4be4-9e8b-2af5baa524fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:51.114774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dee841-5318-4be4-9e8b-2af5baa524fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:51.115061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dee841-5318-4be4-9e8b-2af5baa524fc admin admin] PATH_INFO: `/v3/projects/49048799bbcc4cbfaacaabfbc2ed1178/users/2e82842a4bc540b1904a4c540d5deffe/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:51.131641 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dee841-5318-4be4-9e8b-2af5baa524fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=49048799bbcc4cbfaacaabfbc2ed1178, user_id=2e82842a4bc540b1904a4c540d5deffe, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:51.134759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dee841-5318-4be4-9e8b-2af5baa524fc admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:51.160103 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1225/2443] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:14:51 2022] PUT /identity/v3/projects/49048799bbcc4cbfaacaabfbc2ed1178/users/2e82842a4bc540b1904a4c540d5deffe/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:14:51.168976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551e626-62d1-4a9b-a267-988fd0294cc1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:51.169577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551e626-62d1-4a9b-a267-988fd0294cc1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:51.170042 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5551e626-62d1-4a9b-a267-988fd0294cc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:51.206477 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5551e626-62d1-4a9b-a267-988fd0294cc1 None None] MFA Rules not processed for user `2e82842a4bc540b1904a4c540d5deffe`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:51.208019 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5551e626-62d1-4a9b-a267-988fd0294cc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:51.304027 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1219/2444] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:14:51 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:14:51.313837 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dee841-5318-4be4-9e8b-2af5baa524fc admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:51.316735 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-85dee841-5318-4be4-9e8b-2af5baa524fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:51.396596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dee841-5318-4be4-9e8b-2af5baa524fc admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:51.403064 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e557b72-d33f-4827-ac9e-f20947aa3157 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:51.405008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e557b72-d33f-4827-ac9e-f20947aa3157 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:51.405422 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e557b72-d33f-4827-ac9e-f20947aa3157 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:51.405717 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e557b72-d33f-4827-ac9e-f20947aa3157 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:51.419951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e557b72-d33f-4827-ac9e-f20947aa3157 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:51.421560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e557b72-d33f-4827-ac9e-f20947aa3157 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:51.449075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1226/2445] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:14:51 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:14:52.820612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525d3e4-09eb-4647-8bf1-6c0be1169fd9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:52.821107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525d3e4-09eb-4647-8bf1-6c0be1169fd9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:52.821316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3525d3e4-09eb-4647-8bf1-6c0be1169fd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:52.829320 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:14:52.853646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3525d3e4-09eb-4647-8bf1-6c0be1169fd9 None None] MFA Rules not processed for user `abd62b4b16da4dcd8e0bdbcb6d9db3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:14:52.854939 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3525d3e4-09eb-4647-8bf1-6c0be1169fd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:52.943102 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1220/2446] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:14:52 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:14:52.947795 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e557b72-d33f-4827-ac9e-f20947aa3157 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:52.950486 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8e557b72-d33f-4827-ac9e-f20947aa3157 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:52.972237 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:14:53.022275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e557b72-d33f-4827-ac9e-f20947aa3157 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:53.027505 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252007f-cc63-469a-955f-4c5a7b8a7f61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:53.028230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252007f-cc63-469a-955f-4c5a7b8a7f61 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:53.028546 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252007f-cc63-469a-955f-4c5a7b8a7f61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:53.029053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9252007f-cc63-469a-955f-4c5a7b8a7f61 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:53.029764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252007f-cc63-469a-955f-4c5a7b8a7f61 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:53.031742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9252007f-cc63-469a-955f-4c5a7b8a7f61 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:53.067678 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1227/2447] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:14:52 2022] POST /identity/v3/projects => generated 373 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:14:53.069195 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525d3e4-09eb-4647-8bf1-6c0be1169fd9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:53.081558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3525d3e4-09eb-4647-8bf1-6c0be1169fd9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:53.217775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ee2b7-04e7-4ca8-974a-96f4434b4f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:53.218596 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ee2b7-04e7-4ca8-974a-96f4434b4f17 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:53.218974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ee2b7-04e7-4ca8-974a-96f4434b4f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:53.219322 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a28ee2b7-04e7-4ca8-974a-96f4434b4f17 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:53.220117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ee2b7-04e7-4ca8-974a-96f4434b4f17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:53.222158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a28ee2b7-04e7-4ca8-974a-96f4434b4f17 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:53.268427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1221/2448] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:14:53 2022] POST /identity/v3/users => generated 293 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:14:53.272547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252007f-cc63-469a-955f-4c5a7b8a7f61 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:53.289440 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9252007f-cc63-469a-955f-4c5a7b8a7f61 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:53.304853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302c6a8-8e80-42a6-ae0e-6f0f9824ba69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:53.306201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302c6a8-8e80-42a6-ae0e-6f0f9824ba69 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:53.306201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302c6a8-8e80-42a6-ae0e-6f0f9824ba69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:53.306201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f302c6a8-8e80-42a6-ae0e-6f0f9824ba69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:53.307157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302c6a8-8e80-42a6-ae0e-6f0f9824ba69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:53.309027 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f302c6a8-8e80-42a6-ae0e-6f0f9824ba69 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:53.324177 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1228/2449] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:14:53.328651 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ee2b7-04e7-4ca8-974a-96f4434b4f17 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:53.344966 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a28ee2b7-04e7-4ca8-974a-96f4434b4f17 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:53.352234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c49a3f8-621a-438b-b3b3-e86276a5fb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:53.354601 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49a3f8-621a-438b-b3b3-e86276a5fb4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:53.355332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49a3f8-621a-438b-b3b3-e86276a5fb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:53.355977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c49a3f8-621a-438b-b3b3-e86276a5fb4e admin admin] PATH_INFO: `/v3/projects/99ed0f4b2e7e4ab1a79c30dd247c8061/users/61667a8c1e2c461d8f9891b71ea7a2e4/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:53.395095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c49a3f8-621a-438b-b3b3-e86276a5fb4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ed0f4b2e7e4ab1a79c30dd247c8061, user_id=61667a8c1e2c461d8f9891b71ea7a2e4, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:53.399303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c49a3f8-621a-438b-b3b3-e86276a5fb4e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:53.428242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1222/2450] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:14:53 2022] PUT /identity/v3/projects/99ed0f4b2e7e4ab1a79c30dd247c8061/users/61667a8c1e2c461d8f9891b71ea7a2e4/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 98 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:14:53.430113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302c6a8-8e80-42a6-ae0e-6f0f9824ba69 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:53.449775 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f302c6a8-8e80-42a6-ae0e-6f0f9824ba69 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:53.576811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1f23e-0861-482b-ab3d-9e061332ae25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:53.577642 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1f23e-0861-482b-ab3d-9e061332ae25 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:53.577940 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1f23e-0861-482b-ab3d-9e061332ae25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:53.578358 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d1f23e-0861-482b-ab3d-9e061332ae25 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:53.579132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1f23e-0861-482b-ab3d-9e061332ae25 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:53.585144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d1f23e-0861-482b-ab3d-9e061332ae25 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:53.619310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1229/2451] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:14:53 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:14:53.620276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c49a3f8-621a-438b-b3b3-e86276a5fb4e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:53.634789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c49a3f8-621a-438b-b3b3-e86276a5fb4e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:53.640185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fdd61b3-b988-4600-ae3a-1d6fdf39a7f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:53.641108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd61b3-b988-4600-ae3a-1d6fdf39a7f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:53.641390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd61b3-b988-4600-ae3a-1d6fdf39a7f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:53.641667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fdd61b3-b988-4600-ae3a-1d6fdf39a7f0 admin admin] PATH_INFO: `/v3/projects/99ed0f4b2e7e4ab1a79c30dd247c8061/users/61667a8c1e2c461d8f9891b71ea7a2e4/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:53.645665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd61b3-b988-4600-ae3a-1d6fdf39a7f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=99ed0f4b2e7e4ab1a79c30dd247c8061, user_id=61667a8c1e2c461d8f9891b71ea7a2e4, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:53.648939 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fdd61b3-b988-4600-ae3a-1d6fdf39a7f0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:53.661319 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1223/2452] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:14:53 2022] PUT /identity/v3/projects/99ed0f4b2e7e4ab1a79c30dd247c8061/users/61667a8c1e2c461d8f9891b71ea7a2e4/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:14:58.444827 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1f23e-0861-482b-ab3d-9e061332ae25 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:14:58.446824 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-47d1f23e-0861-482b-ab3d-9e061332ae25 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:58.537734 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d1f23e-0861-482b-ab3d-9e061332ae25 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:14:58.639858 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:14:58.656670 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210705cf-7fb6-458d-a6aa-f739d92eee74 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:14:58.658861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210705cf-7fb6-458d-a6aa-f739d92eee74 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:14:58.660073 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210705cf-7fb6-458d-a6aa-f739d92eee74 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:14:58.660564 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210705cf-7fb6-458d-a6aa-f739d92eee74 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:14:58.667873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-210705cf-7fb6-458d-a6aa-f739d92eee74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:14:58.789723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210705cf-7fb6-458d-a6aa-f739d92eee74 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:14:58.791423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210705cf-7fb6-458d-a6aa-f739d92eee74 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:14:58.828803 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1230/2453] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:14:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:00.457235 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd514cc-7eb4-4bbe-8222-66453461fbb3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:00.457787 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd514cc-7eb4-4bbe-8222-66453461fbb3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:00.457787 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd514cc-7eb4-4bbe-8222-66453461fbb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:00.487328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-dbd514cc-7eb4-4bbe-8222-66453461fbb3 None None] MFA Rules not processed for user `61667a8c1e2c461d8f9891b71ea7a2e4`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:00.488719 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dbd514cc-7eb4-4bbe-8222-66453461fbb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:00.547562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:00.602496 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210705cf-7fb6-458d-a6aa-f739d92eee74 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:00.603776 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-210705cf-7fb6-458d-a6aa-f739d92eee74 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:00.612416 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1224/2454] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:00 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:00.722369 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210705cf-7fb6-458d-a6aa-f739d92eee74 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:00.729169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f543159e-e816-4351-8225-2dc270a42517 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:00.729589 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543159e-e816-4351-8225-2dc270a42517 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:00.729883 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543159e-e816-4351-8225-2dc270a42517 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:00.730161 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f543159e-e816-4351-8225-2dc270a42517 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:00.860989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543159e-e816-4351-8225-2dc270a42517 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:00.864064 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f543159e-e816-4351-8225-2dc270a42517 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:00.881799 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1231/2455] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 280 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:15:01.833137 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd514cc-7eb4-4bbe-8222-66453461fbb3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:01.851312 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd514cc-7eb4-4bbe-8222-66453461fbb3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:01.861970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1ed7a-6388-47df-acbd-d72949478ca7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:01.863343 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1ed7a-6388-47df-acbd-d72949478ca7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:01.863793 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1ed7a-6388-47df-acbd-d72949478ca7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:01.864284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d1ed7a-6388-47df-acbd-d72949478ca7 admin admin] PATH_INFO: `/v3/users/2e82842a4bc540b1904a4c540d5deffe` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:01.866191 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d1ed7a-6388-47df-acbd-d72949478ca7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e82842a4bc540b1904a4c540d5deffe)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:01.868364 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d1ed7a-6388-47df-acbd-d72949478ca7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:01.952325 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-76d1ed7a-6388-47df-acbd-d72949478ca7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e82842a4bc540b1904a4c540d5deffe'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:01.978774 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-76d1ed7a-6388-47df-acbd-d72949478ca7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e82842a4bc540b1904a4c540d5deffe'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:01.988677 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-76d1ed7a-6388-47df-acbd-d72949478ca7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e82842a4bc540b1904a4c540d5deffe'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:01.989694 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-76d1ed7a-6388-47df-acbd-d72949478ca7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e82842a4bc540b1904a4c540d5deffe'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:01.999691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-76d1ed7a-6388-47df-acbd-d72949478ca7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e82842a4bc540b1904a4c540d5deffe'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:02.008933 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1225/2456] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:15:01 2022] DELETE /identity/v3/users/2e82842a4bc540b1904a4c540d5deffe => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:02.014006 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623406e-5463-4ac3-9c6c-9c0c54e5bb98 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:02.014471 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623406e-5463-4ac3-9c6c-9c0c54e5bb98 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:02.014575 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1623406e-5463-4ac3-9c6c-9c0c54e5bb98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:02.036275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-1623406e-5463-4ac3-9c6c-9c0c54e5bb98 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:02.037510 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1623406e-5463-4ac3-9c6c-9c0c54e5bb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:02.071305 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:02.116607 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1232/2457] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:02.126302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1ed7a-6388-47df-acbd-d72949478ca7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:02.127935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-76d1ed7a-6388-47df-acbd-d72949478ca7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:02.231478 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d1ed7a-6388-47df-acbd-d72949478ca7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:02.241792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba8974a-1d41-4dbb-af64-600e52739d34 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:02.243483 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8974a-1d41-4dbb-af64-600e52739d34 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:02.244045 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8974a-1d41-4dbb-af64-600e52739d34 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:02.244501 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba8974a-1d41-4dbb-af64-600e52739d34 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:02.260803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8974a-1d41-4dbb-af64-600e52739d34 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:02.263505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bba8974a-1d41-4dbb-af64-600e52739d34 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:02.275023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1226/2458] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:02.854474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623406e-5463-4ac3-9c6c-9c0c54e5bb98 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:02.856147 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1623406e-5463-4ac3-9c6c-9c0c54e5bb98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:02.903288 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1623406e-5463-4ac3-9c6c-9c0c54e5bb98 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:02.915239 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c0f7a4-f4c0-4043-a551-0d91015aa590 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:02.916512 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0f7a4-f4c0-4043-a551-0d91015aa590 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:02.917538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0f7a4-f4c0-4043-a551-0d91015aa590 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:02.918215 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c0f7a4-f4c0-4043-a551-0d91015aa590 admin admin] PATH_INFO: `/v3/projects/49048799bbcc4cbfaacaabfbc2ed1178` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:02.919868 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0f7a4-f4c0-4043-a551-0d91015aa590 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49048799bbcc4cbfaacaabfbc2ed1178)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:02.923612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c0f7a4-f4c0-4043-a551-0d91015aa590 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:02.995189 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f9c0f7a4-f4c0-4043-a551-0d91015aa590 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49048799bbcc4cbfaacaabfbc2ed1178', 'request_id': 'req-f9c0f7a4-f4c0-4043-a551-0d91015aa590'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:03.005775 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f9c0f7a4-f4c0-4043-a551-0d91015aa590 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49048799bbcc4cbfaacaabfbc2ed1178', 'request_id': 'req-f9c0f7a4-f4c0-4043-a551-0d91015aa590'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:03.019951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f9c0f7a4-f4c0-4043-a551-0d91015aa590 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49048799bbcc4cbfaacaabfbc2ed1178', 'request_id': 'req-f9c0f7a4-f4c0-4043-a551-0d91015aa590'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:03.038004 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f9c0f7a4-f4c0-4043-a551-0d91015aa590 admin admin] The token cache is being invalidate because project 49048799bbcc4cbfaacaabfbc2ed1178 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:15:03.038456 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f9c0f7a4-f4c0-4043-a551-0d91015aa590 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:03.039214 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-f9c0f7a4-f4c0-4043-a551-0d91015aa590 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:03.041385 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1233/2459] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:15:02 2022] DELETE /identity/v3/projects/49048799bbcc4cbfaacaabfbc2ed1178 => generated 0 bytes in 188 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:03.046547 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4601842-9ece-4624-ade3-502ff423e339 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:03.047472 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4601842-9ece-4624-ade3-502ff423e339 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:03.047472 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4601842-9ece-4624-ade3-502ff423e339 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:03.066444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcf44d-267c-44aa-a778-8f87da71d2ed None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:03.067018 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcf44d-267c-44aa-a778-8f87da71d2ed None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:03.067649 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27dcf44d-267c-44aa-a778-8f87da71d2ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:03.077923 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e4601842-9ece-4624-ade3-502ff423e339 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:03.079472 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e4601842-9ece-4624-ade3-502ff423e339 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:03.098414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-27dcf44d-267c-44aa-a778-8f87da71d2ed None None] MFA Rules not processed for user `abd62b4b16da4dcd8e0bdbcb6d9db3a8`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:03.100066 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-27dcf44d-267c-44aa-a778-8f87da71d2ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:03.128698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:03.152559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1227/2460] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:03.162875 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96137287-897c-4b61-a358-876a0838d1b0 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:03.163391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96137287-897c-4b61-a358-876a0838d1b0 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:03.163849 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96137287-897c-4b61-a358-876a0838d1b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:03.164886 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1234/2461] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:03 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:03.195702 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-96137287-897c-4b61-a358-876a0838d1b0 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:03.197182 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-96137287-897c-4b61-a358-876a0838d1b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:03.207364 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1228/2462] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:03.211467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dcf44d-267c-44aa-a778-8f87da71d2ed None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:03.224821 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27dcf44d-267c-44aa-a778-8f87da71d2ed None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:03.230153 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedefbb4-96af-4a36-962b-8b6e1a3d07d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:03.230837 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedefbb4-96af-4a36-962b-8b6e1a3d07d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:03.231190 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedefbb4-96af-4a36-962b-8b6e1a3d07d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:03.231468 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eedefbb4-96af-4a36-962b-8b6e1a3d07d4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:03.232257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedefbb4-96af-4a36-962b-8b6e1a3d07d4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:15:03.232603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedefbb4-96af-4a36-962b-8b6e1a3d07d4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:03.234301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eedefbb4-96af-4a36-962b-8b6e1a3d07d4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:03.248311 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1235/2463] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:15:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:15:03.255252 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96137287-897c-4b61-a358-876a0838d1b0 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:03.264789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96137287-897c-4b61-a358-876a0838d1b0 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:03.269786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a49db7-6f82-4c9e-937c-335eacc87a8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:03.270568 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a49db7-6f82-4c9e-937c-335eacc87a8c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:03.270926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a49db7-6f82-4c9e-937c-335eacc87a8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:03.271273 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0a49db7-6f82-4c9e-937c-335eacc87a8c admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:03.272133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a49db7-6f82-4c9e-937c-335eacc87a8c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:03.274218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0a49db7-6f82-4c9e-937c-335eacc87a8c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:03.317001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1229/2464] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:15:03 2022] POST /identity/v3/projects => generated 373 bytes in 64 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:03.320776 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedefbb4-96af-4a36-962b-8b6e1a3d07d4 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:03.333998 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eedefbb4-96af-4a36-962b-8b6e1a3d07d4 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:03.383919 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:03.401019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef205e-e381-4a1d-93c8-9c6e81766b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:03.402016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef205e-e381-4a1d-93c8-9c6e81766b9a admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:03.402497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef205e-e381-4a1d-93c8-9c6e81766b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:03.402780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03ef205e-e381-4a1d-93c8-9c6e81766b9a admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:03.403490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef205e-e381-4a1d-93c8-9c6e81766b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:03.405519 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03ef205e-e381-4a1d-93c8-9c6e81766b9a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:03.431964 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1236/2465] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:15:03 2022] POST /identity/v3/users => generated 287 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:03.436245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a49db7-6f82-4c9e-937c-335eacc87a8c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:03.451398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0a49db7-6f82-4c9e-937c-335eacc87a8c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:03.456794 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c2397-5b38-4554-bd1c-930259ea7ff0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:03.457499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c2397-5b38-4554-bd1c-930259ea7ff0 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:03.457788 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c2397-5b38-4554-bd1c-930259ea7ff0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:03.458068 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce8c2397-5b38-4554-bd1c-930259ea7ff0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:03.458823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c2397-5b38-4554-bd1c-930259ea7ff0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:03.460514 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce8c2397-5b38-4554-bd1c-930259ea7ff0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:03.475144 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1230/2466] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:03.479848 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef205e-e381-4a1d-93c8-9c6e81766b9a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:03.502784 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03ef205e-e381-4a1d-93c8-9c6e81766b9a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:03.508520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd5765f-c260-4477-8ff4-053ee8ede687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:03.509234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5765f-c260-4477-8ff4-053ee8ede687 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:03.509545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5765f-c260-4477-8ff4-053ee8ede687 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:03.510009 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd5765f-c260-4477-8ff4-053ee8ede687 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:03.510615 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5765f-c260-4477-8ff4-053ee8ede687 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:03.512473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd5765f-c260-4477-8ff4-053ee8ede687 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:03.531898 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1237/2467] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:03 2022] GET /identity/v3/roles => generated 1420 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:03.534491 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c2397-5b38-4554-bd1c-930259ea7ff0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:03.551764 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce8c2397-5b38-4554-bd1c-930259ea7ff0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:03.558291 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc16869-71f5-4dc7-a017-bf35575be95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:03.559583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc16869-71f5-4dc7-a017-bf35575be95c admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:03.559905 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc16869-71f5-4dc7-a017-bf35575be95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:03.560336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc16869-71f5-4dc7-a017-bf35575be95c admin admin] PATH_INFO: `/v3/projects/e7bac61121f6455e8b9cccd259a8da85/users/f88bec9e63f44fa089246749e58ca0df/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:03.579297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc16869-71f5-4dc7-a017-bf35575be95c admin admin] RBAC: Authorizing `identity:create_grant(project_id=e7bac61121f6455e8b9cccd259a8da85, user_id=f88bec9e63f44fa089246749e58ca0df, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:03.585115 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc16869-71f5-4dc7-a017-bf35575be95c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:03.605597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1231/2468] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:03 2022] PUT /identity/v3/projects/e7bac61121f6455e8b9cccd259a8da85/users/f88bec9e63f44fa089246749e58ca0df/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 72 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:03.616471 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609db77-e26c-4307-a908-f74c2f73327e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:03.616779 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609db77-e26c-4307-a908-f74c2f73327e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:03.617071 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4609db77-e26c-4307-a908-f74c2f73327e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:03.680842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4609db77-e26c-4307-a908-f74c2f73327e None None] MFA Rules not processed for user `f88bec9e63f44fa089246749e58ca0df`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:03.682543 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4609db77-e26c-4307-a908-f74c2f73327e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:03.786818 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1238/2469] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:03 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:03.792762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc16869-71f5-4dc7-a017-bf35575be95c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:04.063660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc16869-71f5-4dc7-a017-bf35575be95c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:04.071393 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb36ffb-861c-4939-8e7a-c2b204204781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:04.073292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb36ffb-861c-4939-8e7a-c2b204204781 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:04.073955 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb36ffb-861c-4939-8e7a-c2b204204781 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:04.074442 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb36ffb-861c-4939-8e7a-c2b204204781 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:04.075354 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb36ffb-861c-4939-8e7a-c2b204204781 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:04.078332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eb36ffb-861c-4939-8e7a-c2b204204781 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:04.133153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1232/2470] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:15:03 2022] POST /identity/v3/projects => generated 373 bytes in 341 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:04.137424 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609db77-e26c-4307-a908-f74c2f73327e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:04.165588 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4609db77-e26c-4307-a908-f74c2f73327e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:04.289755 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe957b65-fc25-4b7a-ae9f-5aec7c53d6b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:04.291316 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe957b65-fc25-4b7a-ae9f-5aec7c53d6b6 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:04.291550 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe957b65-fc25-4b7a-ae9f-5aec7c53d6b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:04.291915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe957b65-fc25-4b7a-ae9f-5aec7c53d6b6 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:04.292774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe957b65-fc25-4b7a-ae9f-5aec7c53d6b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:04.311950 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe957b65-fc25-4b7a-ae9f-5aec7c53d6b6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:04.343907 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1239/2471] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:15:04 2022] POST /identity/v3/users => generated 293 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:04.344704 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb36ffb-861c-4939-8e7a-c2b204204781 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:04.362007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb36ffb-861c-4939-8e7a-c2b204204781 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:04.382417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d8a68-d464-4a2f-9c95-708f93d58c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:04.383784 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d8a68-d464-4a2f-9c95-708f93d58c3e admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:04.384432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d8a68-d464-4a2f-9c95-708f93d58c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:04.384875 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68d8a68-d464-4a2f-9c95-708f93d58c3e admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:04.386846 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d8a68-d464-4a2f-9c95-708f93d58c3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:04.393489 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68d8a68-d464-4a2f-9c95-708f93d58c3e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:04.406062 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1233/2472] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:15:04.410534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe957b65-fc25-4b7a-ae9f-5aec7c53d6b6 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:04.431495 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe957b65-fc25-4b7a-ae9f-5aec7c53d6b6 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:04.444983 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94715731-22cf-442a-9564-a1fda3b3d3a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:04.446399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94715731-22cf-442a-9564-a1fda3b3d3a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:04.446882 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94715731-22cf-442a-9564-a1fda3b3d3a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:04.447368 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94715731-22cf-442a-9564-a1fda3b3d3a0 admin admin] PATH_INFO: `/v3/projects/2a9084e3584f44cb844b6af374f6eb71/users/6c731507d7054573a82cc62e9214dfa6/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:04.469911 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94715731-22cf-442a-9564-a1fda3b3d3a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a9084e3584f44cb844b6af374f6eb71, user_id=6c731507d7054573a82cc62e9214dfa6, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:04.474297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94715731-22cf-442a-9564-a1fda3b3d3a0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:04.497974 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1240/2473] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:04 2022] PUT /identity/v3/projects/2a9084e3584f44cb844b6af374f6eb71/users/6c731507d7054573a82cc62e9214dfa6/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 88 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:15:04.502099 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d8a68-d464-4a2f-9c95-708f93d58c3e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:04.515580 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68d8a68-d464-4a2f-9c95-708f93d58c3e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:04.615721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:04.624382 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b1bcdf1-1b94-433e-aef7-054e3a04781c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:04.625365 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bcdf1-1b94-433e-aef7-054e3a04781c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:04.626114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bcdf1-1b94-433e-aef7-054e3a04781c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:04.626698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b1bcdf1-1b94-433e-aef7-054e3a04781c admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:04.627860 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1bcdf1-1b94-433e-aef7-054e3a04781c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:04.630147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b1bcdf1-1b94-433e-aef7-054e3a04781c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:04.643861 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1234/2474] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:04 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:15:04.645697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94715731-22cf-442a-9564-a1fda3b3d3a0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:04.662408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94715731-22cf-442a-9564-a1fda3b3d3a0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:04.668219 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fc1aa-e96f-4f5d-a94d-67a54e403487 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:04.669143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fc1aa-e96f-4f5d-a94d-67a54e403487 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:04.669446 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fc1aa-e96f-4f5d-a94d-67a54e403487 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:04.669720 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5fc1aa-e96f-4f5d-a94d-67a54e403487 admin admin] PATH_INFO: `/v3/projects/2a9084e3584f44cb844b6af374f6eb71/users/6c731507d7054573a82cc62e9214dfa6/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:04.678651 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5fc1aa-e96f-4f5d-a94d-67a54e403487 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a9084e3584f44cb844b6af374f6eb71, user_id=6c731507d7054573a82cc62e9214dfa6, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:04.682300 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5fc1aa-e96f-4f5d-a94d-67a54e403487 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:04.695522 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1241/2475] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:04 2022] PUT /identity/v3/projects/2a9084e3584f44cb844b6af374f6eb71/users/6c731507d7054573a82cc62e9214dfa6/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:04.706258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee358b1a-63d0-4d80-b8b8-16c3f76e2c6a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:04.706567 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee358b1a-63d0-4d80-b8b8-16c3f76e2c6a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:04.706884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee358b1a-63d0-4d80-b8b8-16c3f76e2c6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:04.743184 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fc1aa-e96f-4f5d-a94d-67a54e403487 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:04.745300 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ee5fc1aa-e96f-4f5d-a94d-67a54e403487 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:04.756601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:04.760510 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ee358b1a-63d0-4d80-b8b8-16c3f76e2c6a None None] MFA Rules not processed for user `6c731507d7054573a82cc62e9214dfa6`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:04.761848 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ee358b1a-63d0-4d80-b8b8-16c3f76e2c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:04.827393 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5fc1aa-e96f-4f5d-a94d-67a54e403487 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:04.834786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438f2c7-f113-4e0c-838b-f45e4a8321ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:04.837013 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438f2c7-f113-4e0c-838b-f45e4a8321ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:04.837312 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438f2c7-f113-4e0c-838b-f45e4a8321ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:04.837653 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d438f2c7-f113-4e0c-838b-f45e4a8321ce admin admin] PATH_INFO: `/v3/users/a0289402c6f245839a86b7b66714a8f6` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:04.840669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438f2c7-f113-4e0c-838b-f45e4a8321ce admin admin] RBAC: Authorizing `identity:delete_user(user_id=a0289402c6f245839a86b7b66714a8f6)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:04.843360 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d438f2c7-f113-4e0c-838b-f45e4a8321ce admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:04.847005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1235/2476] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:04 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:04.865101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee358b1a-63d0-4d80-b8b8-16c3f76e2c6a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:04.867405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ee358b1a-63d0-4d80-b8b8-16c3f76e2c6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:04.928706 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d438f2c7-f113-4e0c-838b-f45e4a8321ce admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a0289402c6f245839a86b7b66714a8f6'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:04.930319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d438f2c7-f113-4e0c-838b-f45e4a8321ce admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a0289402c6f245839a86b7b66714a8f6'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:04.950431 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d438f2c7-f113-4e0c-838b-f45e4a8321ce admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a0289402c6f245839a86b7b66714a8f6'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:04.966941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d438f2c7-f113-4e0c-838b-f45e4a8321ce admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a0289402c6f245839a86b7b66714a8f6'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:04.967660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d438f2c7-f113-4e0c-838b-f45e4a8321ce admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a0289402c6f245839a86b7b66714a8f6'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:04.970996 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee358b1a-63d0-4d80-b8b8-16c3f76e2c6a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:04.974987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e25bb-b1f6-45ed-8efa-4db240a9c0c8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:04.975675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e25bb-b1f6-45ed-8efa-4db240a9c0c8 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:04.976076 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e25bb-b1f6-45ed-8efa-4db240a9c0c8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:04.976312 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214e25bb-b1f6-45ed-8efa-4db240a9c0c8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:04.978846 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-214e25bb-b1f6-45ed-8efa-4db240a9c0c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:04.986547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1242/2477] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:15:04 2022] DELETE /identity/v3/users/a0289402c6f245839a86b7b66714a8f6 => generated 0 bytes in 245 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:15:04.994621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e25bb-b1f6-45ed-8efa-4db240a9c0c8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:04.996138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214e25bb-b1f6-45ed-8efa-4db240a9c0c8 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:05.007966 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1236/2478] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:04 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:15:05.099717 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438f2c7-f113-4e0c-838b-f45e4a8321ce admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:05.101325 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d438f2c7-f113-4e0c-838b-f45e4a8321ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:05.182212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d438f2c7-f113-4e0c-838b-f45e4a8321ce admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:05.188543 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e56db-d2c3-49a7-bc85-abf654fc16a4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:05.189614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e56db-d2c3-49a7-bc85-abf654fc16a4 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:05.189935 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e56db-d2c3-49a7-bc85-abf654fc16a4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:05.190470 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7e56db-d2c3-49a7-bc85-abf654fc16a4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:05.194247 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-dd7e56db-d2c3-49a7-bc85-abf654fc16a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:05.232437 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e25bb-b1f6-45ed-8efa-4db240a9c0c8 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:05.234512 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-214e25bb-b1f6-45ed-8efa-4db240a9c0c8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:05.254634 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214e25bb-b1f6-45ed-8efa-4db240a9c0c8 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:05.297025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:05.317387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e56db-d2c3-49a7-bc85-abf654fc16a4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:05.318995 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7e56db-d2c3-49a7-bc85-abf654fc16a4 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:05.330978 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1243/2479] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:05 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 232 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:05.331845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e56db-d2c3-49a7-bc85-abf654fc16a4 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:05.332717 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824cc153-0cd4-44e3-8ac2-1124a0899828 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:05.333684 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cc153-0cd4-44e3-8ac2-1124a0899828 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:05.334034 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cc153-0cd4-44e3-8ac2-1124a0899828 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:05.334378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824cc153-0cd4-44e3-8ac2-1124a0899828 admin admin] PATH_INFO: `/v3/projects/93946fd1575e45428d2388015d64628f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:05.335663 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cc153-0cd4-44e3-8ac2-1124a0899828 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93946fd1575e45428d2388015d64628f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:05.337810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824cc153-0cd4-44e3-8ac2-1124a0899828 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:05.342466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-dd7e56db-d2c3-49a7-bc85-abf654fc16a4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:05.377999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:05.422463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-824cc153-0cd4-44e3-8ac2-1124a0899828 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93946fd1575e45428d2388015d64628f', 'request_id': 'req-824cc153-0cd4-44e3-8ac2-1124a0899828'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:05.434807 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-824cc153-0cd4-44e3-8ac2-1124a0899828 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93946fd1575e45428d2388015d64628f', 'request_id': 'req-824cc153-0cd4-44e3-8ac2-1124a0899828'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:05.440963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7e56db-d2c3-49a7-bc85-abf654fc16a4 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:05.444559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-824cc153-0cd4-44e3-8ac2-1124a0899828 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93946fd1575e45428d2388015d64628f', 'request_id': 'req-824cc153-0cd4-44e3-8ac2-1124a0899828'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:05.464479 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-824cc153-0cd4-44e3-8ac2-1124a0899828 admin admin] The token cache is being invalidate because project 93946fd1575e45428d2388015d64628f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:15:05.464681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-824cc153-0cd4-44e3-8ac2-1124a0899828 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:05.465300 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-824cc153-0cd4-44e3-8ac2-1124a0899828 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:05.467728 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1237/2480] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:15:05 2022] DELETE /identity/v3/projects/93946fd1575e45428d2388015d64628f => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:05.473685 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40467ca9-7878-4cfe-a11c-c835844efe13 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:05.473977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40467ca9-7878-4cfe-a11c-c835844efe13 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:05.474262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40467ca9-7878-4cfe-a11c-c835844efe13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:05.511381 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-40467ca9-7878-4cfe-a11c-c835844efe13 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:05.512695 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-40467ca9-7878-4cfe-a11c-c835844efe13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:05.525841 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebae9a5-6374-45aa-8b27-bd36cc838317 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:05.527116 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebae9a5-6374-45aa-8b27-bd36cc838317 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:05.528581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebae9a5-6374-45aa-8b27-bd36cc838317 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:05.529049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebae9a5-6374-45aa-8b27-bd36cc838317 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:05.531233 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-eebae9a5-6374-45aa-8b27-bd36cc838317 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:05.588619 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1238/2481] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:05.597410 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04f9aa-1d35-4ab3-9db4-0f7e30a0d531 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:05.597716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04f9aa-1d35-4ab3-9db4-0f7e30a0d531 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:05.598134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f04f9aa-1d35-4ab3-9db4-0f7e30a0d531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:05.618625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-8f04f9aa-1d35-4ab3-9db4-0f7e30a0d531 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:05.619890 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-8f04f9aa-1d35-4ab3-9db4-0f7e30a0d531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:05.625542 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebae9a5-6374-45aa-8b27-bd36cc838317 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:05.629154 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebae9a5-6374-45aa-8b27-bd36cc838317 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:05.640557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1239/2482] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:05.644799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04f9aa-1d35-4ab3-9db4-0f7e30a0d531 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:05.657637 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f04f9aa-1d35-4ab3-9db4-0f7e30a0d531 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:05.667721 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1244/2483] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:05.676555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b04ee-2f9f-43b6-bf0e-d3a4b0e25c6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:05.677388 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b04ee-2f9f-43b6-bf0e-d3a4b0e25c6b admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:05.677747 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b04ee-2f9f-43b6-bf0e-d3a4b0e25c6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:05.678098 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20b04ee-2f9f-43b6-bf0e-d3a4b0e25c6b admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:05.683983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b04ee-2f9f-43b6-bf0e-d3a4b0e25c6b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:15:05.684500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b04ee-2f9f-43b6-bf0e-d3a4b0e25c6b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:05.686465 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c20b04ee-2f9f-43b6-bf0e-d3a4b0e25c6b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:05.711216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1240/2484] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:15:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:15:05.715660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebae9a5-6374-45aa-8b27-bd36cc838317 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:05.726604 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebae9a5-6374-45aa-8b27-bd36cc838317 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:05.731538 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d46eb6-c297-496f-b579-834127c7cdf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:05.732274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d46eb6-c297-496f-b579-834127c7cdf2 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:05.732568 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d46eb6-c297-496f-b579-834127c7cdf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:05.732857 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d46eb6-c297-496f-b579-834127c7cdf2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:05.733580 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d46eb6-c297-496f-b579-834127c7cdf2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:05.735531 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d46eb6-c297-496f-b579-834127c7cdf2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:05.802525 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1245/2485] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:15:05 2022] POST /identity/v3/projects => generated 375 bytes in 86 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:05.804162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b04ee-2f9f-43b6-bf0e-d3a4b0e25c6b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:05.817821 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20b04ee-2f9f-43b6-bf0e-d3a4b0e25c6b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:05.930639 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fbf19-41ef-43ed-b0f4-3d994a3ac840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:05.931604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fbf19-41ef-43ed-b0f4-3d994a3ac840 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:05.931851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fbf19-41ef-43ed-b0f4-3d994a3ac840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:05.932263 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-809fbf19-41ef-43ed-b0f4-3d994a3ac840 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:05.933212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809fbf19-41ef-43ed-b0f4-3d994a3ac840 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:05.935889 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-809fbf19-41ef-43ed-b0f4-3d994a3ac840 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:05.971147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1241/2486] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:15:05 2022] POST /identity/v3/users => generated 288 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:05.978608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d46eb6-c297-496f-b579-834127c7cdf2 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:05.993611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d46eb6-c297-496f-b579-834127c7cdf2 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:05.998740 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1b657-ed9e-44ed-be63-38a711f3b237 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:05.999438 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1b657-ed9e-44ed-be63-38a711f3b237 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:05.999738 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1b657-ed9e-44ed-be63-38a711f3b237 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:06.000049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7e1b657-ed9e-44ed-be63-38a711f3b237 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:06.000793 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1b657-ed9e-44ed-be63-38a711f3b237 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:06.002421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7e1b657-ed9e-44ed-be63-38a711f3b237 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:06.019860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1246/2487] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:05 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:06.024731 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fbf19-41ef-43ed-b0f4-3d994a3ac840 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:06.051257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-809fbf19-41ef-43ed-b0f4-3d994a3ac840 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:06.058044 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30af8e31-3809-4a58-a319-464f393a0bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:06.058751 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af8e31-3809-4a58-a319-464f393a0bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:06.059100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af8e31-3809-4a58-a319-464f393a0bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:06.060140 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30af8e31-3809-4a58-a319-464f393a0bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:06.063070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af8e31-3809-4a58-a319-464f393a0bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:06.065337 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30af8e31-3809-4a58-a319-464f393a0bf9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:06.089481 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1242/2488] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:06 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:15:06.095179 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1b657-ed9e-44ed-be63-38a711f3b237 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:06.117404 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7e1b657-ed9e-44ed-be63-38a711f3b237 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:06.122783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9f9e1-49bd-4506-a4ab-d967fffef6bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:06.123702 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9f9e1-49bd-4506-a4ab-d967fffef6bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:06.124023 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9f9e1-49bd-4506-a4ab-d967fffef6bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:06.124302 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c9f9e1-49bd-4506-a4ab-d967fffef6bc admin admin] PATH_INFO: `/v3/projects/26396353fb464a1ca0f0a7c272d1fe40/users/2047a52f45e7423691d0fb226e7f94fd/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:06.146218 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:06.157662 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9f9e1-49bd-4506-a4ab-d967fffef6bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=26396353fb464a1ca0f0a7c272d1fe40, user_id=2047a52f45e7423691d0fb226e7f94fd, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:06.161138 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c9f9e1-49bd-4506-a4ab-d967fffef6bc admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:06.190948 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1247/2489] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:06 2022] PUT /identity/v3/projects/26396353fb464a1ca0f0a7c272d1fe40/users/2047a52f45e7423691d0fb226e7f94fd/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:06.201908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23854c50-a6d4-40a3-a947-0f3f2a7338fa None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:06.202793 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23854c50-a6d4-40a3-a947-0f3f2a7338fa None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:06.203097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23854c50-a6d4-40a3-a947-0f3f2a7338fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:06.225487 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:06.235093 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-23854c50-a6d4-40a3-a947-0f3f2a7338fa None None] MFA Rules not processed for user `2047a52f45e7423691d0fb226e7f94fd`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:06.238125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-23854c50-a6d4-40a3-a947-0f3f2a7338fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:06.349982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1243/2490] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:06.362065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9f9e1-49bd-4506-a4ab-d967fffef6bc admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:06.364170 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c3c9f9e1-49bd-4506-a4ab-d967fffef6bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:06.460862 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c9f9e1-49bd-4506-a4ab-d967fffef6bc admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:06.467809 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d84105-7098-4de7-af6e-6b8d92d90184 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:06.468374 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d84105-7098-4de7-af6e-6b8d92d90184 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:06.468736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d84105-7098-4de7-af6e-6b8d92d90184 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:06.469113 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d84105-7098-4de7-af6e-6b8d92d90184 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:06.480173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d84105-7098-4de7-af6e-6b8d92d90184 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:06.481229 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d84105-7098-4de7-af6e-6b8d92d90184 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:06.491005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1248/2491] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:06.787372 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23854c50-a6d4-40a3-a947-0f3f2a7338fa None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:06.962955 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23854c50-a6d4-40a3-a947-0f3f2a7338fa None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:06.983332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed050983-c09d-4dde-9711-abb508e3b2bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:06.984570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed050983-c09d-4dde-9711-abb508e3b2bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:06.987497 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed050983-c09d-4dde-9711-abb508e3b2bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:06.987983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed050983-c09d-4dde-9711-abb508e3b2bc admin admin] PATH_INFO: `/v3/users/2047a52f45e7423691d0fb226e7f94fd` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:06.989733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed050983-c09d-4dde-9711-abb508e3b2bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=2047a52f45e7423691d0fb226e7f94fd)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:06.993703 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed050983-c09d-4dde-9711-abb508e3b2bc admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:07.070119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ed050983-c09d-4dde-9711-abb508e3b2bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2047a52f45e7423691d0fb226e7f94fd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:07.105417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ed050983-c09d-4dde-9711-abb508e3b2bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2047a52f45e7423691d0fb226e7f94fd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:07.133656 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ed050983-c09d-4dde-9711-abb508e3b2bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2047a52f45e7423691d0fb226e7f94fd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:07.135134 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ed050983-c09d-4dde-9711-abb508e3b2bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2047a52f45e7423691d0fb226e7f94fd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:07.150412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-ed050983-c09d-4dde-9711-abb508e3b2bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2047a52f45e7423691d0fb226e7f94fd'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:07.159090 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1244/2492] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:15:06 2022] DELETE /identity/v3/users/2047a52f45e7423691d0fb226e7f94fd => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:07.166739 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cc8b2-de4d-4a82-91c6-5f1f903baf61 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:07.167041 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cc8b2-de4d-4a82-91c6-5f1f903baf61 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:07.167332 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8cc8b2-de4d-4a82-91c6-5f1f903baf61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:07.200941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8b8cc8b2-de4d-4a82-91c6-5f1f903baf61 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:07.203129 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8b8cc8b2-de4d-4a82-91c6-5f1f903baf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:07.314665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1249/2493] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:07.325785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed050983-c09d-4dde-9711-abb508e3b2bc admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:07.329513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ed050983-c09d-4dde-9711-abb508e3b2bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:07.355884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:07.409441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed050983-c09d-4dde-9711-abb508e3b2bc admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:07.413578 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c52ccbfa-8549-4c0e-af54-a190bb2d94ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:07.414295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ccbfa-8549-4c0e-af54-a190bb2d94ce service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:07.414594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ccbfa-8549-4c0e-af54-a190bb2d94ce service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:07.414878 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c52ccbfa-8549-4c0e-af54-a190bb2d94ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:07.425851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ccbfa-8549-4c0e-af54-a190bb2d94ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:07.427664 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c52ccbfa-8549-4c0e-af54-a190bb2d94ce service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:07.439178 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1245/2494] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:07.751313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cc8b2-de4d-4a82-91c6-5f1f903baf61 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:07.753419 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8b8cc8b2-de4d-4a82-91c6-5f1f903baf61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:07.766704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8cc8b2-de4d-4a82-91c6-5f1f903baf61 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:07.771552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296de38-eb49-4a42-be5d-de2c370ca879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:07.772638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296de38-eb49-4a42-be5d-de2c370ca879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:07.772935 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296de38-eb49-4a42-be5d-de2c370ca879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:07.773242 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296de38-eb49-4a42-be5d-de2c370ca879 admin admin] PATH_INFO: `/v3/projects/26396353fb464a1ca0f0a7c272d1fe40` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:07.774536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296de38-eb49-4a42-be5d-de2c370ca879 admin admin] RBAC: Authorizing `identity:delete_project(project_id=26396353fb464a1ca0f0a7c272d1fe40)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:07.776471 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296de38-eb49-4a42-be5d-de2c370ca879 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:07.861368 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b296de38-eb49-4a42-be5d-de2c370ca879 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '26396353fb464a1ca0f0a7c272d1fe40', 'request_id': 'req-b296de38-eb49-4a42-be5d-de2c370ca879'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:07.867147 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b296de38-eb49-4a42-be5d-de2c370ca879 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '26396353fb464a1ca0f0a7c272d1fe40', 'request_id': 'req-b296de38-eb49-4a42-be5d-de2c370ca879'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:07.878766 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b296de38-eb49-4a42-be5d-de2c370ca879 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '26396353fb464a1ca0f0a7c272d1fe40', 'request_id': 'req-b296de38-eb49-4a42-be5d-de2c370ca879'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:07.894855 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:07.910338 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b296de38-eb49-4a42-be5d-de2c370ca879 admin admin] The token cache is being invalidate because project 26396353fb464a1ca0f0a7c272d1fe40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:15:07.910824 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b296de38-eb49-4a42-be5d-de2c370ca879 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:07.911678 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b296de38-eb49-4a42-be5d-de2c370ca879 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:07.914877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1250/2495] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:15:07 2022] DELETE /identity/v3/projects/26396353fb464a1ca0f0a7c272d1fe40 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:07.919812 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cdd62-0226-4c63-8491-0a690b07d63e None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:07.920149 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cdd62-0226-4c63-8491-0a690b07d63e None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:07.920433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956cdd62-0226-4c63-8491-0a690b07d63e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:07.943278 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-956cdd62-0226-4c63-8491-0a690b07d63e None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:07.946048 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-956cdd62-0226-4c63-8491-0a690b07d63e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:08.092411 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1246/2496] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:08.100521 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3d97b-1166-4b3a-99fd-7250c3419a7f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:08.100819 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3d97b-1166-4b3a-99fd-7250c3419a7f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:08.101097 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36c3d97b-1166-4b3a-99fd-7250c3419a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:08.119774 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-36c3d97b-1166-4b3a-99fd-7250c3419a7f None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:08.121446 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-36c3d97b-1166-4b3a-99fd-7250c3419a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:08.130560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1251/2497] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:08.135346 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cdd62-0226-4c63-8491-0a690b07d63e None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:08.151759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956cdd62-0226-4c63-8491-0a690b07d63e None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:08.157711 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dcc85-9c9b-45ed-99aa-d9c1c66cc7ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:08.159047 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dcc85-9c9b-45ed-99aa-d9c1c66cc7ef admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:08.159316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dcc85-9c9b-45ed-99aa-d9c1c66cc7ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:08.159754 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-179dcc85-9c9b-45ed-99aa-d9c1c66cc7ef admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:08.160832 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179dcc85-9c9b-45ed-99aa-d9c1c66cc7ef admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:15:08.161220 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179dcc85-9c9b-45ed-99aa-d9c1c66cc7ef admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:08.163253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-179dcc85-9c9b-45ed-99aa-d9c1c66cc7ef admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:08.181178 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1247/2498] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:15:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:15:08.184470 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3d97b-1166-4b3a-99fd-7250c3419a7f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:08.196145 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36c3d97b-1166-4b3a-99fd-7250c3419a7f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:08.200998 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d520c-ce3c-484b-abc4-17e6e421497e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:08.201696 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d520c-ce3c-484b-abc4-17e6e421497e admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:08.202011 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d520c-ce3c-484b-abc4-17e6e421497e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:08.202291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37d520c-ce3c-484b-abc4-17e6e421497e admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:08.202999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d520c-ce3c-484b-abc4-17e6e421497e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:08.205002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37d520c-ce3c-484b-abc4-17e6e421497e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:08.234763 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1252/2499] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:15:08 2022] POST /identity/v3/projects => generated 355 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:15:08.239052 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dcc85-9c9b-45ed-99aa-d9c1c66cc7ef admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:08.249758 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-179dcc85-9c9b-45ed-99aa-d9c1c66cc7ef admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:08.325210 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fe874-a78a-4370-923c-607c9c007d51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:08.325961 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fe874-a78a-4370-923c-607c9c007d51 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:08.326840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fe874-a78a-4370-923c-607c9c007d51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:08.326840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426fe874-a78a-4370-923c-607c9c007d51 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:08.327207 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426fe874-a78a-4370-923c-607c9c007d51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:08.330268 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426fe874-a78a-4370-923c-607c9c007d51 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:08.363288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1248/2500] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:15:08 2022] POST /identity/v3/users => generated 278 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:08.368036 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d520c-ce3c-484b-abc4-17e6e421497e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:08.388170 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37d520c-ce3c-484b-abc4-17e6e421497e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:08.397508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b30f5-75df-4951-ae62-4425e6df03ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:08.399200 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b30f5-75df-4951-ae62-4425e6df03ec admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:08.399501 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b30f5-75df-4951-ae62-4425e6df03ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:08.399789 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729b30f5-75df-4951-ae62-4425e6df03ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:08.400805 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b30f5-75df-4951-ae62-4425e6df03ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:08.402469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729b30f5-75df-4951-ae62-4425e6df03ec admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:08.425039 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1253/2501] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:08.429045 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fe874-a78a-4370-923c-607c9c007d51 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:08.441417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426fe874-a78a-4370-923c-607c9c007d51 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:08.447599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1f1e22-0866-43d1-8a1f-03465e5bd1f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:08.448643 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f1e22-0866-43d1-8a1f-03465e5bd1f6 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:08.449001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f1e22-0866-43d1-8a1f-03465e5bd1f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:08.449390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1f1e22-0866-43d1-8a1f-03465e5bd1f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:08.450206 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f1e22-0866-43d1-8a1f-03465e5bd1f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:08.452193 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1f1e22-0866-43d1-8a1f-03465e5bd1f6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:08.466080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1249/2502] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:08.470033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b30f5-75df-4951-ae62-4425e6df03ec admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:08.481410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729b30f5-75df-4951-ae62-4425e6df03ec admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:08.490667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17d6ee-5c25-48cd-bc4e-2c64046adc29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:08.491701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17d6ee-5c25-48cd-bc4e-2c64046adc29 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:08.492135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17d6ee-5c25-48cd-bc4e-2c64046adc29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:08.492780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e17d6ee-5c25-48cd-bc4e-2c64046adc29 admin admin] PATH_INFO: `/v3/projects/3796dc34098747fabe0ae777a41a0551/users/4cc5e4c08912428388d36fcd2424a07d/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:08.512424 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17d6ee-5c25-48cd-bc4e-2c64046adc29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3796dc34098747fabe0ae777a41a0551, user_id=4cc5e4c08912428388d36fcd2424a07d, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:08.515365 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e17d6ee-5c25-48cd-bc4e-2c64046adc29 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:08.538815 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1254/2503] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:08 2022] PUT /identity/v3/projects/3796dc34098747fabe0ae777a41a0551/users/4cc5e4c08912428388d36fcd2424a07d/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:08.550399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d43eab-c3b2-4fb2-974a-c3bb25a30f0f None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:08.550976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d43eab-c3b2-4fb2-974a-c3bb25a30f0f None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:08.551326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d43eab-c3b2-4fb2-974a-c3bb25a30f0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:08.583898 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-28d43eab-c3b2-4fb2-974a-c3bb25a30f0f None None] MFA Rules not processed for user `4cc5e4c08912428388d36fcd2424a07d`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:08.585295 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-28d43eab-c3b2-4fb2-974a-c3bb25a30f0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:08.604722 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:08.683695 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1250/2504] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:08 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:08.688249 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17d6ee-5c25-48cd-bc4e-2c64046adc29 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:08.768221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e17d6ee-5c25-48cd-bc4e-2c64046adc29 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:08.773118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49eeb4-649e-4c18-9b15-7c808b70379b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:08.773478 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49eeb4-649e-4c18-9b15-7c808b70379b admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:08.773869 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49eeb4-649e-4c18-9b15-7c808b70379b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:08.774174 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce49eeb4-649e-4c18-9b15-7c808b70379b admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:08.774796 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49eeb4-649e-4c18-9b15-7c808b70379b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:08.777022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce49eeb4-649e-4c18-9b15-7c808b70379b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:08.829956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1255/2505] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:15:08 2022] POST /identity/v3/projects => generated 355 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:08.834094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d43eab-c3b2-4fb2-974a-c3bb25a30f0f None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:08.848240 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d43eab-c3b2-4fb2-974a-c3bb25a30f0f None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:08.926010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e61f3d-1a35-40b6-9069-8cb5a3d1bd9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:08.926791 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e61f3d-1a35-40b6-9069-8cb5a3d1bd9e admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:08.927154 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e61f3d-1a35-40b6-9069-8cb5a3d1bd9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:08.927502 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e61f3d-1a35-40b6-9069-8cb5a3d1bd9e admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:08.929073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e61f3d-1a35-40b6-9069-8cb5a3d1bd9e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:08.931025 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e61f3d-1a35-40b6-9069-8cb5a3d1bd9e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:08.960549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1251/2506] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:15:08 2022] POST /identity/v3/users => generated 284 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:08.964552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49eeb4-649e-4c18-9b15-7c808b70379b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:08.975910 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce49eeb4-649e-4c18-9b15-7c808b70379b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:08.981625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44655c3f-13f8-489a-8f6d-5b276945af06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:08.982423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44655c3f-13f8-489a-8f6d-5b276945af06 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:08.982787 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44655c3f-13f8-489a-8f6d-5b276945af06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:08.983142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44655c3f-13f8-489a-8f6d-5b276945af06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:08.983981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44655c3f-13f8-489a-8f6d-5b276945af06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:08.985680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44655c3f-13f8-489a-8f6d-5b276945af06 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:09.004103 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1256/2507] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:08 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:09.004413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e61f3d-1a35-40b6-9069-8cb5a3d1bd9e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:09.015169 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e61f3d-1a35-40b6-9069-8cb5a3d1bd9e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:09.021146 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281d945-bfc0-4c78-9fda-cbf70cdc803c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:09.022192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281d945-bfc0-4c78-9fda-cbf70cdc803c admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:09.022591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281d945-bfc0-4c78-9fda-cbf70cdc803c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:09.022972 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3281d945-bfc0-4c78-9fda-cbf70cdc803c admin admin] PATH_INFO: `/v3/projects/01be1da020174168a8bef1a3f1a433e3/users/3d665dee2c3e4650b1807a1eeab8f57a/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:09.054930 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281d945-bfc0-4c78-9fda-cbf70cdc803c admin admin] RBAC: Authorizing `identity:create_grant(project_id=01be1da020174168a8bef1a3f1a433e3, user_id=3d665dee2c3e4650b1807a1eeab8f57a, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:09.058454 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3281d945-bfc0-4c78-9fda-cbf70cdc803c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:09.087549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1252/2508] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:09 2022] PUT /identity/v3/projects/01be1da020174168a8bef1a3f1a433e3/users/3d665dee2c3e4650b1807a1eeab8f57a/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 84 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:09.091421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44655c3f-13f8-489a-8f6d-5b276945af06 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:09.107298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44655c3f-13f8-489a-8f6d-5b276945af06 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:09.198685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1513ef70-4fe7-4683-bfd2-b8a0ccd0ecd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:09.198685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513ef70-4fe7-4683-bfd2-b8a0ccd0ecd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:09.198685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513ef70-4fe7-4683-bfd2-b8a0ccd0ecd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:09.198685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1513ef70-4fe7-4683-bfd2-b8a0ccd0ecd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:09.199408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513ef70-4fe7-4683-bfd2-b8a0ccd0ecd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:09.200129 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1513ef70-4fe7-4683-bfd2-b8a0ccd0ecd5 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:09.213997 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1257/2509] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:09 2022] GET /identity/v3/roles => generated 1420 bytes in 123 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:09.218042 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281d945-bfc0-4c78-9fda-cbf70cdc803c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:09.235459 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3281d945-bfc0-4c78-9fda-cbf70cdc803c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:09.240616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5c627-cd26-42c9-81f6-57af560e05c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:09.241533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5c627-cd26-42c9-81f6-57af560e05c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:09.241824 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5c627-cd26-42c9-81f6-57af560e05c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:09.242108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec5c627-cd26-42c9-81f6-57af560e05c2 admin admin] PATH_INFO: `/v3/projects/01be1da020174168a8bef1a3f1a433e3/users/3d665dee2c3e4650b1807a1eeab8f57a/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:09.244183 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5c627-cd26-42c9-81f6-57af560e05c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01be1da020174168a8bef1a3f1a433e3, user_id=3d665dee2c3e4650b1807a1eeab8f57a, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:09.246903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec5c627-cd26-42c9-81f6-57af560e05c2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:09.258173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1253/2510] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:09 2022] PUT /identity/v3/projects/01be1da020174168a8bef1a3f1a433e3/users/3d665dee2c3e4650b1807a1eeab8f57a/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:09.270586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63fd03-9087-4e49-b5f0-612fd82c861d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:09.271254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63fd03-9087-4e49-b5f0-612fd82c861d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:09.271739 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b63fd03-9087-4e49-b5f0-612fd82c861d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:09.291467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:09.325143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-3b63fd03-9087-4e49-b5f0-612fd82c861d None None] MFA Rules not processed for user `3d665dee2c3e4650b1807a1eeab8f57a`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:09.326430 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3b63fd03-9087-4e49-b5f0-612fd82c861d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:09.360793 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5c627-cd26-42c9-81f6-57af560e05c2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:09.362786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9ec5c627-cd26-42c9-81f6-57af560e05c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:09.418713 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1258/2511] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:09 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:09.430553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63fd03-9087-4e49-b5f0-612fd82c861d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:09.431962 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3b63fd03-9087-4e49-b5f0-612fd82c861d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:09.441383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec5c627-cd26-42c9-81f6-57af560e05c2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:09.446625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacaf153-cc5d-40eb-a981-7517eefac503 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:09.447519 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacaf153-cc5d-40eb-a981-7517eefac503 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:09.447808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacaf153-cc5d-40eb-a981-7517eefac503 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:09.448113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacaf153-cc5d-40eb-a981-7517eefac503 admin admin] PATH_INFO: `/v3/users/be5c51fdd2fe4159a83a560dc074da6b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:09.449308 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacaf153-cc5d-40eb-a981-7517eefac503 admin admin] RBAC: Authorizing `identity:delete_user(user_id=be5c51fdd2fe4159a83a560dc074da6b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:09.451246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacaf153-cc5d-40eb-a981-7517eefac503 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:09.471616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:09.497399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b63fd03-9087-4e49-b5f0-612fd82c861d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:09.501431 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97f55c-228d-4d71-a57c-9af05c60e84a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:09.502104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97f55c-228d-4d71-a57c-9af05c60e84a service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:09.502397 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97f55c-228d-4d71-a57c-9af05c60e84a service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:09.502799 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac97f55c-228d-4d71-a57c-9af05c60e84a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:09.513404 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97f55c-228d-4d71-a57c-9af05c60e84a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:09.514088 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dacaf153-cc5d-40eb-a981-7517eefac503 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'be5c51fdd2fe4159a83a560dc074da6b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:09.514861 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac97f55c-228d-4d71-a57c-9af05c60e84a service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:09.524622 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1259/2512] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:09 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:15:09.526165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dacaf153-cc5d-40eb-a981-7517eefac503 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'be5c51fdd2fe4159a83a560dc074da6b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:09.534381 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dacaf153-cc5d-40eb-a981-7517eefac503 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'be5c51fdd2fe4159a83a560dc074da6b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:09.535685 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dacaf153-cc5d-40eb-a981-7517eefac503 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'be5c51fdd2fe4159a83a560dc074da6b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:09.543498 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-dacaf153-cc5d-40eb-a981-7517eefac503 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'be5c51fdd2fe4159a83a560dc074da6b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:09.549853 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1254/2513] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:15:09 2022] DELETE /identity/v3/users/be5c51fdd2fe4159a83a560dc074da6b => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:09.603409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97f55c-228d-4d71-a57c-9af05c60e84a service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:09.612970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac97f55c-228d-4d71-a57c-9af05c60e84a service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:09.666233 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafc8985-cbf0-44ee-8649-e4f7a7928988 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:09.666913 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc8985-cbf0-44ee-8649-e4f7a7928988 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:09.667201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc8985-cbf0-44ee-8649-e4f7a7928988 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:09.667615 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafc8985-cbf0-44ee-8649-e4f7a7928988 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:09.669487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-eafc8985-cbf0-44ee-8649-e4f7a7928988 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:09.722363 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:09.736696 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacaf153-cc5d-40eb-a981-7517eefac503 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:09.738598 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-dacaf153-cc5d-40eb-a981-7517eefac503 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:09.738726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc8985-cbf0-44ee-8649-e4f7a7928988 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:09.740100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafc8985-cbf0-44ee-8649-e4f7a7928988 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:09.750563 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1260/2514] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:09 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:09.828368 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacaf153-cc5d-40eb-a981-7517eefac503 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:09.835211 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413e278e-48c8-4fb3-a2db-4bcc3836da07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:09.836465 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e278e-48c8-4fb3-a2db-4bcc3836da07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:09.836798 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e278e-48c8-4fb3-a2db-4bcc3836da07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:09.837131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413e278e-48c8-4fb3-a2db-4bcc3836da07 admin admin] PATH_INFO: `/v3/projects/feb4d81e1722439591065849449aeb18` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:09.838919 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e278e-48c8-4fb3-a2db-4bcc3836da07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=feb4d81e1722439591065849449aeb18)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:09.841458 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413e278e-48c8-4fb3-a2db-4bcc3836da07 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:09.899288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-413e278e-48c8-4fb3-a2db-4bcc3836da07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'feb4d81e1722439591065849449aeb18', 'request_id': 'req-413e278e-48c8-4fb3-a2db-4bcc3836da07'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:09.916314 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-413e278e-48c8-4fb3-a2db-4bcc3836da07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'feb4d81e1722439591065849449aeb18', 'request_id': 'req-413e278e-48c8-4fb3-a2db-4bcc3836da07'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:09.929350 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-413e278e-48c8-4fb3-a2db-4bcc3836da07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'feb4d81e1722439591065849449aeb18', 'request_id': 'req-413e278e-48c8-4fb3-a2db-4bcc3836da07'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:09.950665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-413e278e-48c8-4fb3-a2db-4bcc3836da07 admin admin] The token cache is being invalidate because project feb4d81e1722439591065849449aeb18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:15:09.951018 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-413e278e-48c8-4fb3-a2db-4bcc3836da07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:09.951776 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-413e278e-48c8-4fb3-a2db-4bcc3836da07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:09.954327 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1255/2515] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:15:09 2022] DELETE /identity/v3/projects/feb4d81e1722439591065849449aeb18 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:15:09.959443 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f743a5-7b96-4784-99be-9351492c3b49 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:09.959743 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f743a5-7b96-4784-99be-9351492c3b49 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:09.960061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f743a5-7b96-4784-99be-9351492c3b49 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:09.978211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d5f743a5-7b96-4784-99be-9351492c3b49 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:09.979427 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d5f743a5-7b96-4784-99be-9351492c3b49 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:10.036976 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1261/2516] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:10.045017 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b171e52-26da-4e3c-846d-6660de70a729 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:10.045346 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b171e52-26da-4e3c-846d-6660de70a729 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:10.045633 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b171e52-26da-4e3c-846d-6660de70a729 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:10.065453 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3b171e52-26da-4e3c-846d-6660de70a729 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:10.066675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3b171e52-26da-4e3c-846d-6660de70a729 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:10.075223 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1256/2517] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:10.079670 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f743a5-7b96-4784-99be-9351492c3b49 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:10.090298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f743a5-7b96-4784-99be-9351492c3b49 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:10.098508 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812dc9c-14f4-4153-805a-23db0c0ca767 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:10.099422 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812dc9c-14f4-4153-805a-23db0c0ca767 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:10.099777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812dc9c-14f4-4153-805a-23db0c0ca767 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:10.100225 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b812dc9c-14f4-4153-805a-23db0c0ca767 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:10.101089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812dc9c-14f4-4153-805a-23db0c0ca767 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:15:10.101403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812dc9c-14f4-4153-805a-23db0c0ca767 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:10.103155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b812dc9c-14f4-4153-805a-23db0c0ca767 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:10.116149 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1262/2518] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:15:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:15:10.120967 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b171e52-26da-4e3c-846d-6660de70a729 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:10.131129 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b171e52-26da-4e3c-846d-6660de70a729 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:10.136921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b471e-225e-4be2-8a14-b6ca9fbda4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:10.137617 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b471e-225e-4be2-8a14-b6ca9fbda4cf admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:10.137912 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b471e-225e-4be2-8a14-b6ca9fbda4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:10.138196 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b471e-225e-4be2-8a14-b6ca9fbda4cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:10.138945 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b471e-225e-4be2-8a14-b6ca9fbda4cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:10.140961 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14b471e-225e-4be2-8a14-b6ca9fbda4cf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:10.151667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:10.171604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1257/2519] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:15:10 2022] POST /identity/v3/projects => generated 403 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:10.176139 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812dc9c-14f4-4153-805a-23db0c0ca767 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:10.186319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b812dc9c-14f4-4153-805a-23db0c0ca767 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:10.243556 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5d760-444a-468a-97d6-7ef54be046da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:10.244280 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5d760-444a-468a-97d6-7ef54be046da admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:10.244540 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5d760-444a-468a-97d6-7ef54be046da admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:10.244820 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f5d760-444a-468a-97d6-7ef54be046da admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:10.245519 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5d760-444a-468a-97d6-7ef54be046da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:10.247481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f5d760-444a-468a-97d6-7ef54be046da admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:10.270492 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1263/2520] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:15:10 2022] POST /identity/v3/users => generated 302 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:10.276114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b471e-225e-4be2-8a14-b6ca9fbda4cf admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:10.286561 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b471e-225e-4be2-8a14-b6ca9fbda4cf admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:10.291536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11cbe57-befe-424b-996c-381a389e0130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:10.292173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cbe57-befe-424b-996c-381a389e0130 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:10.292460 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cbe57-befe-424b-996c-381a389e0130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:10.292744 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b11cbe57-befe-424b-996c-381a389e0130 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:10.293477 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11cbe57-befe-424b-996c-381a389e0130 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:10.295092 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b11cbe57-befe-424b-996c-381a389e0130 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:10.306776 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1258/2521] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:10.311406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5d760-444a-468a-97d6-7ef54be046da admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:10.327493 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f5d760-444a-468a-97d6-7ef54be046da admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:10.337332 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c5e23e-2e96-4e18-8024-cdad7d606758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:10.338050 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5e23e-2e96-4e18-8024-cdad7d606758 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:10.338341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5e23e-2e96-4e18-8024-cdad7d606758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:10.338609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c5e23e-2e96-4e18-8024-cdad7d606758 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:10.339375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c5e23e-2e96-4e18-8024-cdad7d606758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:10.341142 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c5e23e-2e96-4e18-8024-cdad7d606758 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:10.359047 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1264/2522] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:10 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:10.365457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11cbe57-befe-424b-996c-381a389e0130 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:10.376987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b11cbe57-befe-424b-996c-381a389e0130 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:10.383243 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945655fc-4376-4091-b0b6-d2484ab28e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:10.384215 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945655fc-4376-4091-b0b6-d2484ab28e57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:10.384510 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945655fc-4376-4091-b0b6-d2484ab28e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:10.384794 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-945655fc-4376-4091-b0b6-d2484ab28e57 admin admin] PATH_INFO: `/v3/projects/bc49293e4ad34e58a7d1bb4919d9586e/users/e8e4ad94a8714c0ea54b211da76bb869/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:10.400545 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945655fc-4376-4091-b0b6-d2484ab28e57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bc49293e4ad34e58a7d1bb4919d9586e, user_id=e8e4ad94a8714c0ea54b211da76bb869, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:10.403223 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-945655fc-4376-4091-b0b6-d2484ab28e57 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:10.419850 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1259/2523] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:10 2022] PUT /identity/v3/projects/bc49293e4ad34e58a7d1bb4919d9586e/users/e8e4ad94a8714c0ea54b211da76bb869/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:10.426524 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b12129-5076-4534-a190-dbeb9e8a6d55 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:10.426951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b12129-5076-4534-a190-dbeb9e8a6d55 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:10.427363 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b12129-5076-4534-a190-dbeb9e8a6d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:10.446317 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-26b12129-5076-4534-a190-dbeb9e8a6d55 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:10.447529 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-26b12129-5076-4534-a190-dbeb9e8a6d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:10.495432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:10.516534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1265/2524] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:10.527731 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945655fc-4376-4091-b0b6-d2484ab28e57 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:10.529818 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-945655fc-4376-4091-b0b6-d2484ab28e57 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:10.593561 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-945655fc-4376-4091-b0b6-d2484ab28e57 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:10.598250 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb92157-2745-44ec-9bfb-9bc4551c6592 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:10.599007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb92157-2745-44ec-9bfb-9bc4551c6592 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:10.599322 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb92157-2745-44ec-9bfb-9bc4551c6592 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:10.599604 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb92157-2745-44ec-9bfb-9bc4551c6592 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:10.609439 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb92157-2745-44ec-9bfb-9bc4551c6592 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:10.610893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb92157-2745-44ec-9bfb-9bc4551c6592 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:10.620502 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1260/2525] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:11.265564 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa14a7-e324-4498-b222-e983df5096b0 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:11.266099 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa14a7-e324-4498-b222-e983df5096b0 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:11.266230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa14a7-e324-4498-b222-e983df5096b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:11.292896 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-29aa14a7-e324-4498-b222-e983df5096b0 None None] MFA Rules not processed for user `e8e4ad94a8714c0ea54b211da76bb869`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:11.294093 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-29aa14a7-e324-4498-b222-e983df5096b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:11.364018 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1266/2526] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:11 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:11.368366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb92157-2745-44ec-9bfb-9bc4551c6592 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:11.378055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb92157-2745-44ec-9bfb-9bc4551c6592 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:11.383001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070da2e2-f394-44eb-843d-b15da77f59be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:11.383645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070da2e2-f394-44eb-843d-b15da77f59be admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:11.383973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070da2e2-f394-44eb-843d-b15da77f59be admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:11.384255 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-070da2e2-f394-44eb-843d-b15da77f59be admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:11.384950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070da2e2-f394-44eb-843d-b15da77f59be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:11.386870 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-070da2e2-f394-44eb-843d-b15da77f59be admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:11.415697 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1261/2527] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:15:11 2022] POST /identity/v3/projects => generated 405 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:11.417135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa14a7-e324-4498-b222-e983df5096b0 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:11.429114 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa14a7-e324-4498-b222-e983df5096b0 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:11.489057 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476eff1a-25c1-49b2-89e7-818d88e72561 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:11.489743 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476eff1a-25c1-49b2-89e7-818d88e72561 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:11.490033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476eff1a-25c1-49b2-89e7-818d88e72561 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:11.490365 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-476eff1a-25c1-49b2-89e7-818d88e72561 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:11.491115 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476eff1a-25c1-49b2-89e7-818d88e72561 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:11.493150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-476eff1a-25c1-49b2-89e7-818d88e72561 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:11.515858 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1267/2528] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:15:11 2022] POST /identity/v3/users => generated 309 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:11.520029 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070da2e2-f394-44eb-843d-b15da77f59be admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:11.531030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-070da2e2-f394-44eb-843d-b15da77f59be admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:11.536091 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10060eac-892b-4e34-8c84-31cb5a645c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:11.536947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10060eac-892b-4e34-8c84-31cb5a645c34 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:11.537296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10060eac-892b-4e34-8c84-31cb5a645c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:11.537718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10060eac-892b-4e34-8c84-31cb5a645c34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:11.538621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10060eac-892b-4e34-8c84-31cb5a645c34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:11.540434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10060eac-892b-4e34-8c84-31cb5a645c34 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:11.551316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1262/2529] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:11.555522 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476eff1a-25c1-49b2-89e7-818d88e72561 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:11.565884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-476eff1a-25c1-49b2-89e7-818d88e72561 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:11.571288 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf73100-cc10-470f-82cb-54bb1c878e81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:11.572228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf73100-cc10-470f-82cb-54bb1c878e81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:11.572517 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf73100-cc10-470f-82cb-54bb1c878e81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:11.572801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf73100-cc10-470f-82cb-54bb1c878e81 admin admin] PATH_INFO: `/v3/projects/7b80aca7f5db47a3b2c147faa0e8f9ea/users/eb32cbf69bcc4db7a8a628386d6f6d5d/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:11.588383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf73100-cc10-470f-82cb-54bb1c878e81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b80aca7f5db47a3b2c147faa0e8f9ea, user_id=eb32cbf69bcc4db7a8a628386d6f6d5d, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:11.591086 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf73100-cc10-470f-82cb-54bb1c878e81 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:11.606475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1268/2530] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:11 2022] PUT /identity/v3/projects/7b80aca7f5db47a3b2c147faa0e8f9ea/users/eb32cbf69bcc4db7a8a628386d6f6d5d/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:11.610447 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10060eac-892b-4e34-8c84-31cb5a645c34 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:11.620832 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10060eac-892b-4e34-8c84-31cb5a645c34 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:11.680298 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-635ba82a-2648-49f1-bb3a-bb41cf2896ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:11.681009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ba82a-2648-49f1-bb3a-bb41cf2896ac admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:11.681302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ba82a-2648-49f1-bb3a-bb41cf2896ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:11.681588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-635ba82a-2648-49f1-bb3a-bb41cf2896ac admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:11.682384 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ba82a-2648-49f1-bb3a-bb41cf2896ac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:11.684186 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-635ba82a-2648-49f1-bb3a-bb41cf2896ac admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:11.695768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1263/2531] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:11 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:11.699845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf73100-cc10-470f-82cb-54bb1c878e81 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:11.712263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf73100-cc10-470f-82cb-54bb1c878e81 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:11.717817 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f367c-25f8-4f64-b946-feb8590f2823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:11.718811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f367c-25f8-4f64-b946-feb8590f2823 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:11.719145 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f367c-25f8-4f64-b946-feb8590f2823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:11.719429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30f367c-25f8-4f64-b946-feb8590f2823 admin admin] PATH_INFO: `/v3/projects/7b80aca7f5db47a3b2c147faa0e8f9ea/users/eb32cbf69bcc4db7a8a628386d6f6d5d/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:11.721535 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f367c-25f8-4f64-b946-feb8590f2823 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b80aca7f5db47a3b2c147faa0e8f9ea, user_id=eb32cbf69bcc4db7a8a628386d6f6d5d, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:11.724278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30f367c-25f8-4f64-b946-feb8590f2823 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:11.736828 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1269/2532] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:11 2022] PUT /identity/v3/projects/7b80aca7f5db47a3b2c147faa0e8f9ea/users/eb32cbf69bcc4db7a8a628386d6f6d5d/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:12.530611 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba8da8-808a-48f0-91fd-ddcc1b258271 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:12.531192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba8da8-808a-48f0-91fd-ddcc1b258271 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:12.531321 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ba8da8-808a-48f0-91fd-ddcc1b258271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:12.572484 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-24ba8da8-808a-48f0-91fd-ddcc1b258271 None None] MFA Rules not processed for user `eb32cbf69bcc4db7a8a628386d6f6d5d`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:12.573861 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-24ba8da8-808a-48f0-91fd-ddcc1b258271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:12.592114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:12.669876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1264/2533] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:12 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:13.709450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f367c-25f8-4f64-b946-feb8590f2823 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:13.767238 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:13.813453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30f367c-25f8-4f64-b946-feb8590f2823 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:13.819746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640f294-f1db-490f-ac84-3f5a143dbb18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:13.820724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640f294-f1db-490f-ac84-3f5a143dbb18 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:13.821047 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640f294-f1db-490f-ac84-3f5a143dbb18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:13.821404 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b640f294-f1db-490f-ac84-3f5a143dbb18 admin admin] PATH_INFO: `/v3/users/e8e4ad94a8714c0ea54b211da76bb869` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:13.822643 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640f294-f1db-490f-ac84-3f5a143dbb18 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8e4ad94a8714c0ea54b211da76bb869)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:13.824625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b640f294-f1db-490f-ac84-3f5a143dbb18 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:13.886927 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b640f294-f1db-490f-ac84-3f5a143dbb18 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8e4ad94a8714c0ea54b211da76bb869'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:13.887607 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b640f294-f1db-490f-ac84-3f5a143dbb18 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8e4ad94a8714c0ea54b211da76bb869'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:13.900139 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b640f294-f1db-490f-ac84-3f5a143dbb18 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8e4ad94a8714c0ea54b211da76bb869'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:13.917405 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b640f294-f1db-490f-ac84-3f5a143dbb18 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8e4ad94a8714c0ea54b211da76bb869'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:13.918528 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b640f294-f1db-490f-ac84-3f5a143dbb18 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8e4ad94a8714c0ea54b211da76bb869'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:13.944014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1270/2534] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:15:13 2022] DELETE /identity/v3/users/e8e4ad94a8714c0ea54b211da76bb869 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:14.148893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba8da8-808a-48f0-91fd-ddcc1b258271 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:14.173031 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ba8da8-808a-48f0-91fd-ddcc1b258271 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:14.239214 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170a56d-3c78-4f7f-b516-59e7d2b6807a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:14.240128 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170a56d-3c78-4f7f-b516-59e7d2b6807a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:14.240423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170a56d-3c78-4f7f-b516-59e7d2b6807a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:14.240811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d170a56d-3c78-4f7f-b516-59e7d2b6807a admin admin] PATH_INFO: `/v3/projects/bc49293e4ad34e58a7d1bb4919d9586e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:14.242521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170a56d-3c78-4f7f-b516-59e7d2b6807a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bc49293e4ad34e58a7d1bb4919d9586e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:14.244586 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d170a56d-3c78-4f7f-b516-59e7d2b6807a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:14.306427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d170a56d-3c78-4f7f-b516-59e7d2b6807a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bc49293e4ad34e58a7d1bb4919d9586e', 'request_id': 'req-d170a56d-3c78-4f7f-b516-59e7d2b6807a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:14.314975 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d170a56d-3c78-4f7f-b516-59e7d2b6807a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bc49293e4ad34e58a7d1bb4919d9586e', 'request_id': 'req-d170a56d-3c78-4f7f-b516-59e7d2b6807a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:14.321484 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d170a56d-3c78-4f7f-b516-59e7d2b6807a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bc49293e4ad34e58a7d1bb4919d9586e', 'request_id': 'req-d170a56d-3c78-4f7f-b516-59e7d2b6807a'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:14.331811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d170a56d-3c78-4f7f-b516-59e7d2b6807a admin admin] The token cache is being invalidate because project bc49293e4ad34e58a7d1bb4919d9586e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:15:14.332141 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d170a56d-3c78-4f7f-b516-59e7d2b6807a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:14.332808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d170a56d-3c78-4f7f-b516-59e7d2b6807a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:14.334401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1265/2535] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:15:14 2022] DELETE /identity/v3/projects/bc49293e4ad34e58a7d1bb4919d9586e => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:15:14.338889 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640f294-f1db-490f-ac84-3f5a143dbb18 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:14.340464 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b640f294-f1db-490f-ac84-3f5a143dbb18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:14.355026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b640f294-f1db-490f-ac84-3f5a143dbb18 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:14.428102 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db31bac-92fa-41a1-a274-483d377fac81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:14.429008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db31bac-92fa-41a1-a274-483d377fac81 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:14.429428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db31bac-92fa-41a1-a274-483d377fac81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:14.429827 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db31bac-92fa-41a1-a274-483d377fac81 admin admin] PATH_INFO: `/v3/users/eb32cbf69bcc4db7a8a628386d6f6d5d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:14.431003 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db31bac-92fa-41a1-a274-483d377fac81 admin admin] RBAC: Authorizing `identity:delete_user(user_id=eb32cbf69bcc4db7a8a628386d6f6d5d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:14.433012 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db31bac-92fa-41a1-a274-483d377fac81 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:14.483594 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9db31bac-92fa-41a1-a274-483d377fac81 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'eb32cbf69bcc4db7a8a628386d6f6d5d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:14.484341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9db31bac-92fa-41a1-a274-483d377fac81 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'eb32cbf69bcc4db7a8a628386d6f6d5d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:14.491900 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=81,keystone:DELETE=18,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:14.498157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9db31bac-92fa-41a1-a274-483d377fac81 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'eb32cbf69bcc4db7a8a628386d6f6d5d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:14.508881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9db31bac-92fa-41a1-a274-483d377fac81 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'eb32cbf69bcc4db7a8a628386d6f6d5d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:14.509606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9db31bac-92fa-41a1-a274-483d377fac81 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'eb32cbf69bcc4db7a8a628386d6f6d5d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:14.523488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1271/2536] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:15:14 2022] DELETE /identity/v3/users/eb32cbf69bcc4db7a8a628386d6f6d5d => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:14.706646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170a56d-3c78-4f7f-b516-59e7d2b6807a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:14.708163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d170a56d-3c78-4f7f-b516-59e7d2b6807a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:14.722562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d170a56d-3c78-4f7f-b516-59e7d2b6807a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:14.760348 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:14.785988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8920353e-702c-42aa-98b1-2d19b491c6c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:14.786874 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920353e-702c-42aa-98b1-2d19b491c6c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:14.787165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920353e-702c-42aa-98b1-2d19b491c6c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:14.787443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8920353e-702c-42aa-98b1-2d19b491c6c0 admin admin] PATH_INFO: `/v3/projects/7b80aca7f5db47a3b2c147faa0e8f9ea` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:14.788753 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920353e-702c-42aa-98b1-2d19b491c6c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b80aca7f5db47a3b2c147faa0e8f9ea)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:14.790838 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8920353e-702c-42aa-98b1-2d19b491c6c0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:14.838482 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8920353e-702c-42aa-98b1-2d19b491c6c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b80aca7f5db47a3b2c147faa0e8f9ea', 'request_id': 'req-8920353e-702c-42aa-98b1-2d19b491c6c0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:14.847717 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8920353e-702c-42aa-98b1-2d19b491c6c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b80aca7f5db47a3b2c147faa0e8f9ea', 'request_id': 'req-8920353e-702c-42aa-98b1-2d19b491c6c0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:14.854427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8920353e-702c-42aa-98b1-2d19b491c6c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b80aca7f5db47a3b2c147faa0e8f9ea', 'request_id': 'req-8920353e-702c-42aa-98b1-2d19b491c6c0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:14.864927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8920353e-702c-42aa-98b1-2d19b491c6c0 admin admin] The token cache is being invalidate because project 7b80aca7f5db47a3b2c147faa0e8f9ea was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:15:14.865219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8920353e-702c-42aa-98b1-2d19b491c6c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:14.865872 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8920353e-702c-42aa-98b1-2d19b491c6c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:14.867493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1266/2537] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:15:14 2022] DELETE /identity/v3/projects/7b80aca7f5db47a3b2c147faa0e8f9ea => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:14.872591 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03b716-1f67-435b-8087-0e2f9492c2c8 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:14.873088 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03b716-1f67-435b-8087-0e2f9492c2c8 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:14.873205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f03b716-1f67-435b-8087-0e2f9492c2c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:14.891575 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4f03b716-1f67-435b-8087-0e2f9492c2c8 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:14.892877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4f03b716-1f67-435b-8087-0e2f9492c2c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:14.964444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1272/2538] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:14.972407 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce8c49-6b58-4c1d-85e3-889a0bea3cd2 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:14.972704 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce8c49-6b58-4c1d-85e3-889a0bea3cd2 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:14.972989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5ce8c49-6b58-4c1d-85e3-889a0bea3cd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:14.992609 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-c5ce8c49-6b58-4c1d-85e3-889a0bea3cd2 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:14.993829 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c5ce8c49-6b58-4c1d-85e3-889a0bea3cd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:15.002766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1267/2539] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:15.007592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03b716-1f67-435b-8087-0e2f9492c2c8 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:15.018001 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f03b716-1f67-435b-8087-0e2f9492c2c8 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:15.023677 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3638b-1d24-4aa6-97f5-1638ebebed2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:15.024415 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3638b-1d24-4aa6-97f5-1638ebebed2e admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:15.024705 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3638b-1d24-4aa6-97f5-1638ebebed2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:15.024950 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f3638b-1d24-4aa6-97f5-1638ebebed2e admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:15.025773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3638b-1d24-4aa6-97f5-1638ebebed2e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:15:15.026094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3638b-1d24-4aa6-97f5-1638ebebed2e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:15.027855 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f3638b-1d24-4aa6-97f5-1638ebebed2e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:15.043419 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1273/2540] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:15:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:15:15.048278 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce8c49-6b58-4c1d-85e3-889a0bea3cd2 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:15.058086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5ce8c49-6b58-4c1d-85e3-889a0bea3cd2 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:15.063597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6f67a-29bb-496a-9f29-eb4307b319b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:15.064323 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6f67a-29bb-496a-9f29-eb4307b319b5 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:15.067781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6f67a-29bb-496a-9f29-eb4307b319b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:15.067781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e6f67a-29bb-496a-9f29-eb4307b319b5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:15.067781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6f67a-29bb-496a-9f29-eb4307b319b5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:15.067781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e6f67a-29bb-496a-9f29-eb4307b319b5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:15.097190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1268/2541] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:15:15 2022] POST /identity/v3/projects => generated 385 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:15.101263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3638b-1d24-4aa6-97f5-1638ebebed2e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:15.111335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f3638b-1d24-4aa6-97f5-1638ebebed2e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:15.171415 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f3d45-3fda-482a-8375-186448c2e04e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:15.171634 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f3d45-3fda-482a-8375-186448c2e04e admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:15.171867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f3d45-3fda-482a-8375-186448c2e04e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:15.172194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-286f3d45-3fda-482a-8375-186448c2e04e admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:15.172941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f3d45-3fda-482a-8375-186448c2e04e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:15.175055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-286f3d45-3fda-482a-8375-186448c2e04e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:15.197229 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1274/2542] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:15:15 2022] POST /identity/v3/users => generated 293 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:15:15.203015 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6f67a-29bb-496a-9f29-eb4307b319b5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:15.216343 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e6f67a-29bb-496a-9f29-eb4307b319b5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:15.222387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2c5cc-45ae-42d0-84a1-c73eb2bf81c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:15.223224 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2c5cc-45ae-42d0-84a1-c73eb2bf81c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:15.223517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2c5cc-45ae-42d0-84a1-c73eb2bf81c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:15.223798 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80f2c5cc-45ae-42d0-84a1-c73eb2bf81c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:15.224572 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2c5cc-45ae-42d0-84a1-c73eb2bf81c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:15.226301 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80f2c5cc-45ae-42d0-84a1-c73eb2bf81c3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:15.237354 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1269/2543] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:15.241734 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f3d45-3fda-482a-8375-186448c2e04e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:15.255450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-286f3d45-3fda-482a-8375-186448c2e04e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:15.259486 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806fd32a-c8e2-46f9-8241-e25987b2f477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:15.260236 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fd32a-c8e2-46f9-8241-e25987b2f477 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:15.260598 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fd32a-c8e2-46f9-8241-e25987b2f477 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:15.260885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806fd32a-c8e2-46f9-8241-e25987b2f477 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:15.261759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806fd32a-c8e2-46f9-8241-e25987b2f477 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:15.263487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806fd32a-c8e2-46f9-8241-e25987b2f477 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:15.275017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1275/2544] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:15.279248 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2c5cc-45ae-42d0-84a1-c73eb2bf81c3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:15.290340 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80f2c5cc-45ae-42d0-84a1-c73eb2bf81c3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:15.295521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba43f5d-8739-46ac-bc9b-86fc878090f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:15.296685 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba43f5d-8739-46ac-bc9b-86fc878090f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:15.297163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba43f5d-8739-46ac-bc9b-86fc878090f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:15.297624 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ba43f5d-8739-46ac-bc9b-86fc878090f2 admin admin] PATH_INFO: `/v3/projects/ca090846b79e4879b9643ce0e1925a5f/users/32b318b13791489983af97faa71c3dc4/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:15.318372 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba43f5d-8739-46ac-bc9b-86fc878090f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ca090846b79e4879b9643ce0e1925a5f, user_id=32b318b13791489983af97faa71c3dc4, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:15.321234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ba43f5d-8739-46ac-bc9b-86fc878090f2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:15.335778 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1270/2545] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:15 2022] PUT /identity/v3/projects/ca090846b79e4879b9643ce0e1925a5f/users/32b318b13791489983af97faa71c3dc4/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:15.347589 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444bb0e7-8ee1-484c-870a-edecb4d5a622 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:15.347951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444bb0e7-8ee1-484c-870a-edecb4d5a622 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:15.348381 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444bb0e7-8ee1-484c-870a-edecb4d5a622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:15.375589 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-444bb0e7-8ee1-484c-870a-edecb4d5a622 None None] MFA Rules not processed for user `32b318b13791489983af97faa71c3dc4`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:15.377011 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-444bb0e7-8ee1-484c-870a-edecb4d5a622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:15.412150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:15.456575 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1276/2546] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:15.462711 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba43f5d-8739-46ac-bc9b-86fc878090f2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:15.534755 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ba43f5d-8739-46ac-bc9b-86fc878090f2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:15.539485 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ba207-5c95-473e-a805-0828d2945123 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:15.540197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ba207-5c95-473e-a805-0828d2945123 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:15.540488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ba207-5c95-473e-a805-0828d2945123 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:15.540791 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d77ba207-5c95-473e-a805-0828d2945123 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:15.541471 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ba207-5c95-473e-a805-0828d2945123 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:15.543585 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d77ba207-5c95-473e-a805-0828d2945123 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:15.552414 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:15.571292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1271/2547] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:15:15 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:15.575274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444bb0e7-8ee1-484c-870a-edecb4d5a622 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:15.586302 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444bb0e7-8ee1-484c-870a-edecb4d5a622 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:15.647845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e9cbe-3934-4ca7-9e3b-0622b37cea84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:15.648585 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e9cbe-3934-4ca7-9e3b-0622b37cea84 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:15.648878 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e9cbe-3934-4ca7-9e3b-0622b37cea84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:15.649164 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8e9cbe-3934-4ca7-9e3b-0622b37cea84 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:15.649871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e9cbe-3934-4ca7-9e3b-0622b37cea84 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:15.651813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8e9cbe-3934-4ca7-9e3b-0622b37cea84 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:15.674139 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1277/2548] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:15:15 2022] POST /identity/v3/users => generated 293 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:15:15.678311 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ba207-5c95-473e-a805-0828d2945123 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:15.689539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d77ba207-5c95-473e-a805-0828d2945123 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:15.695375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d27f2a-6a15-41e7-b053-94100940d046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:15.696316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d27f2a-6a15-41e7-b053-94100940d046 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:15.696726 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d27f2a-6a15-41e7-b053-94100940d046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:15.697014 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d27f2a-6a15-41e7-b053-94100940d046 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:15.697773 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d27f2a-6a15-41e7-b053-94100940d046 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:15.699473 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d27f2a-6a15-41e7-b053-94100940d046 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:15.713244 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1272/2549] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:15.717465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e9cbe-3934-4ca7-9e3b-0622b37cea84 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:15.728172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8e9cbe-3934-4ca7-9e3b-0622b37cea84 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:15.734144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d7f23d2-e57b-49ba-9912-a4230f333eda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:15.734813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f23d2-e57b-49ba-9912-a4230f333eda admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:15.735103 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f23d2-e57b-49ba-9912-a4230f333eda admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:15.735389 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d7f23d2-e57b-49ba-9912-a4230f333eda admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:15.736158 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f23d2-e57b-49ba-9912-a4230f333eda admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:15.737775 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d7f23d2-e57b-49ba-9912-a4230f333eda admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:15.748432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1278/2550] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:15 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:15.752647 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d27f2a-6a15-41e7-b053-94100940d046 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:15.763767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d27f2a-6a15-41e7-b053-94100940d046 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:15.769486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72dc17-43ba-42a3-b8e9-d5f694da30b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:15.770404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72dc17-43ba-42a3-b8e9-d5f694da30b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:15.770701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72dc17-43ba-42a3-b8e9-d5f694da30b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:15.770987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c72dc17-43ba-42a3-b8e9-d5f694da30b3 admin admin] PATH_INFO: `/v3/projects/7ce5cd4a15e8466784af0a4d93f4d0fd/users/5a4374e9425845b19b001c2271f2fe4f/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:15.787163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72dc17-43ba-42a3-b8e9-d5f694da30b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7ce5cd4a15e8466784af0a4d93f4d0fd, user_id=5a4374e9425845b19b001c2271f2fe4f, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:15.789897 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c72dc17-43ba-42a3-b8e9-d5f694da30b3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:15.805819 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1273/2551] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:15 2022] PUT /identity/v3/projects/7ce5cd4a15e8466784af0a4d93f4d0fd/users/5a4374e9425845b19b001c2271f2fe4f/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:15:15.816293 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5893b15-9783-4735-8a1e-e195a1695a61 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:15.816640 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5893b15-9783-4735-8a1e-e195a1695a61 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:15.817012 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5893b15-9783-4735-8a1e-e195a1695a61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:15.848172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-a5893b15-9783-4735-8a1e-e195a1695a61 None None] MFA Rules not processed for user `5a4374e9425845b19b001c2271f2fe4f`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:15.849457 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a5893b15-9783-4735-8a1e-e195a1695a61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:15.913708 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1279/2552] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:15.918529 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72dc17-43ba-42a3-b8e9-d5f694da30b3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:15.988543 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c72dc17-43ba-42a3-b8e9-d5f694da30b3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:15.994887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247eb537-f3eb-4788-933c-ec55def44965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:15.995633 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247eb537-f3eb-4788-933c-ec55def44965 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:15.995936 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247eb537-f3eb-4788-933c-ec55def44965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:15.996245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-247eb537-f3eb-4788-933c-ec55def44965 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:15.996953 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247eb537-f3eb-4788-933c-ec55def44965 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:15.998974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-247eb537-f3eb-4788-933c-ec55def44965 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:16.028405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1274/2553] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:15:15 2022] POST /identity/v3/projects => generated 385 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:16.032619 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5893b15-9783-4735-8a1e-e195a1695a61 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:16.043575 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5893b15-9783-4735-8a1e-e195a1695a61 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:16.077410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:16.102921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00f9e5-4af8-457e-9131-4f590f310e4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:16.103602 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00f9e5-4af8-457e-9131-4f590f310e4c admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:16.103924 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00f9e5-4af8-457e-9131-4f590f310e4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:16.104204 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec00f9e5-4af8-457e-9131-4f590f310e4c admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:16.104901 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00f9e5-4af8-457e-9131-4f590f310e4c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:16.106853 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec00f9e5-4af8-457e-9131-4f590f310e4c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:16.129408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1280/2554] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:15:16 2022] POST /identity/v3/users => generated 299 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:16.133584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247eb537-f3eb-4788-933c-ec55def44965 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:16.144042 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-247eb537-f3eb-4788-933c-ec55def44965 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:16.148868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf1f14-4983-4a4a-8b62-c76c56a9806c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:16.149533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf1f14-4983-4a4a-8b62-c76c56a9806c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:16.149817 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf1f14-4983-4a4a-8b62-c76c56a9806c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:16.150097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cf1f14-4983-4a4a-8b62-c76c56a9806c admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:16.150830 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf1f14-4983-4a4a-8b62-c76c56a9806c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:16.152497 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cf1f14-4983-4a4a-8b62-c76c56a9806c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:16.163364 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1275/2555] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:16.167766 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00f9e5-4af8-457e-9131-4f590f310e4c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:16.180128 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec00f9e5-4af8-457e-9131-4f590f310e4c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:16.186127 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad87c20-d246-4c72-a00a-2b3ddfbf9de0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:16.187061 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad87c20-d246-4c72-a00a-2b3ddfbf9de0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:16.187353 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad87c20-d246-4c72-a00a-2b3ddfbf9de0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:16.187640 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad87c20-d246-4c72-a00a-2b3ddfbf9de0 admin admin] PATH_INFO: `/v3/projects/11ccf6bcfaa8411aa2ffb6cb59c90bc6/users/b217a1c8d81046beb00b11fdd37af035/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:16.204695 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad87c20-d246-4c72-a00a-2b3ddfbf9de0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11ccf6bcfaa8411aa2ffb6cb59c90bc6, user_id=b217a1c8d81046beb00b11fdd37af035, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:16.207386 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ad87c20-d246-4c72-a00a-2b3ddfbf9de0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:16.223481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1281/2556] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:16 2022] PUT /identity/v3/projects/11ccf6bcfaa8411aa2ffb6cb59c90bc6/users/b217a1c8d81046beb00b11fdd37af035/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:16.227790 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf1f14-4983-4a4a-8b62-c76c56a9806c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:16.238354 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cf1f14-4983-4a4a-8b62-c76c56a9806c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:16.294678 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55edcf69-009f-44cc-9a87-e1916ec891e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:16.295268 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edcf69-009f-44cc-9a87-e1916ec891e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:16.295554 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edcf69-009f-44cc-9a87-e1916ec891e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:16.295859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55edcf69-009f-44cc-9a87-e1916ec891e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:16.296588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55edcf69-009f-44cc-9a87-e1916ec891e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:16.298204 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55edcf69-009f-44cc-9a87-e1916ec891e6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:16.308574 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1276/2557] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:16 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:16.312908 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad87c20-d246-4c72-a00a-2b3ddfbf9de0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:16.327805 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad87c20-d246-4c72-a00a-2b3ddfbf9de0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:16.334058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406bd9f-8887-410b-bcec-26f7bda92fe8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:16.335078 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406bd9f-8887-410b-bcec-26f7bda92fe8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:16.335450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406bd9f-8887-410b-bcec-26f7bda92fe8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:16.335894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2406bd9f-8887-410b-bcec-26f7bda92fe8 admin admin] PATH_INFO: `/v3/projects/11ccf6bcfaa8411aa2ffb6cb59c90bc6/users/b217a1c8d81046beb00b11fdd37af035/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:16.338423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406bd9f-8887-410b-bcec-26f7bda92fe8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11ccf6bcfaa8411aa2ffb6cb59c90bc6, user_id=b217a1c8d81046beb00b11fdd37af035, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:16.341172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2406bd9f-8887-410b-bcec-26f7bda92fe8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:16.353492 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1282/2558] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:16 2022] PUT /identity/v3/projects/11ccf6bcfaa8411aa2ffb6cb59c90bc6/users/b217a1c8d81046beb00b11fdd37af035/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:16.364926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8896091-d3c4-44a0-958d-567f49fc3264 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:16.365191 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8896091-d3c4-44a0-958d-567f49fc3264 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:16.365485 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8896091-d3c4-44a0-958d-567f49fc3264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:16.397028 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f8896091-d3c4-44a0-958d-567f49fc3264 None None] MFA Rules not processed for user `b217a1c8d81046beb00b11fdd37af035`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:16.398688 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f8896091-d3c4-44a0-958d-567f49fc3264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:16.412251 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:16.483953 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1277/2559] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:16 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:16.500502 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406bd9f-8887-410b-bcec-26f7bda92fe8 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:16.502545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2406bd9f-8887-410b-bcec-26f7bda92fe8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:16.569271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2406bd9f-8887-410b-bcec-26f7bda92fe8 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:16.573121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04057019-f5ec-4a0e-9983-505c3534f73a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:16.573838 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04057019-f5ec-4a0e-9983-505c3534f73a service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:16.574132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04057019-f5ec-4a0e-9983-505c3534f73a service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:16.574431 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04057019-f5ec-4a0e-9983-505c3534f73a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:16.584269 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04057019-f5ec-4a0e-9983-505c3534f73a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:16.585725 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04057019-f5ec-4a0e-9983-505c3534f73a service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:16.595483 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1283/2560] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:16 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:17.188865 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8896091-d3c4-44a0-958d-567f49fc3264 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:17.190372 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f8896091-d3c4-44a0-958d-567f49fc3264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:17.203768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8896091-d3c4-44a0-958d-567f49fc3264 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:17.207745 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691943b-baf5-4276-9eb8-0bcf2a88b239 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:17.208474 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691943b-baf5-4276-9eb8-0bcf2a88b239 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:17.208762 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691943b-baf5-4276-9eb8-0bcf2a88b239 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:17.209045 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9691943b-baf5-4276-9eb8-0bcf2a88b239 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:17.275678 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691943b-baf5-4276-9eb8-0bcf2a88b239 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:17.277184 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9691943b-baf5-4276-9eb8-0bcf2a88b239 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:17.286616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1278/2561] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:17 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:21.358970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04057019-f5ec-4a0e-9983-505c3534f73a service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:21.367990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04057019-f5ec-4a0e-9983-505c3534f73a service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:21.371647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66fb010-5dbe-4032-8c71-888fedf42a01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:21.372324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fb010-5dbe-4032-8c71-888fedf42a01 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:21.372610 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fb010-5dbe-4032-8c71-888fedf42a01 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:21.372892 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66fb010-5dbe-4032-8c71-888fedf42a01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:21.441678 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66fb010-5dbe-4032-8c71-888fedf42a01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:21.444089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66fb010-5dbe-4032-8c71-888fedf42a01 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:21.455100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1284/2562] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:21 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:23.728975 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691943b-baf5-4276-9eb8-0bcf2a88b239 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:23.791170 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9691943b-baf5-4276-9eb8-0bcf2a88b239 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:23.795672 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a7669-0a44-4a24-9210-eacc583c82d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:23.796596 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a7669-0a44-4a24-9210-eacc583c82d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:23.796885 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a7669-0a44-4a24-9210-eacc583c82d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:23.797168 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d8a7669-0a44-4a24-9210-eacc583c82d2 admin admin] PATH_INFO: `/v3/users/32b318b13791489983af97faa71c3dc4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:23.798358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a7669-0a44-4a24-9210-eacc583c82d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=32b318b13791489983af97faa71c3dc4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:23.800349 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d8a7669-0a44-4a24-9210-eacc583c82d2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:23.856775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9d8a7669-0a44-4a24-9210-eacc583c82d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '32b318b13791489983af97faa71c3dc4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:23.858805 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:23.870429 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9d8a7669-0a44-4a24-9210-eacc583c82d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '32b318b13791489983af97faa71c3dc4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:23.879706 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9d8a7669-0a44-4a24-9210-eacc583c82d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '32b318b13791489983af97faa71c3dc4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:23.880373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9d8a7669-0a44-4a24-9210-eacc583c82d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '32b318b13791489983af97faa71c3dc4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:23.887964 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-9d8a7669-0a44-4a24-9210-eacc583c82d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '32b318b13791489983af97faa71c3dc4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:23.894274 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1279/2563] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:15:23 2022] DELETE /identity/v3/users/32b318b13791489983af97faa71c3dc4 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:15:23.899010 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1addd6-6f36-4bd4-8e96-7e1fec80d56f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:23.899495 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1addd6-6f36-4bd4-8e96-7e1fec80d56f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:23.899614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e1addd6-6f36-4bd4-8e96-7e1fec80d56f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:23.919316 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4e1addd6-6f36-4bd4-8e96-7e1fec80d56f None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:23.920642 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4e1addd6-6f36-4bd4-8e96-7e1fec80d56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:23.949274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:23.987716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1285/2564] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:23.997980 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a7669-0a44-4a24-9210-eacc583c82d2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:23.999677 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9d8a7669-0a44-4a24-9210-eacc583c82d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:24.066260 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d8a7669-0a44-4a24-9210-eacc583c82d2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:24.071598 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88892e4-68b2-42f1-8068-8dd6477cacab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:24.072412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88892e4-68b2-42f1-8068-8dd6477cacab service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:24.072820 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88892e4-68b2-42f1-8068-8dd6477cacab service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:24.073218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88892e4-68b2-42f1-8068-8dd6477cacab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:24.091449 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88892e4-68b2-42f1-8068-8dd6477cacab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:24.093163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88892e4-68b2-42f1-8068-8dd6477cacab service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:24.106056 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1280/2565] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:24.603455 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1addd6-6f36-4bd4-8e96-7e1fec80d56f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:24.605842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4e1addd6-6f36-4bd4-8e96-7e1fec80d56f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:24.622329 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e1addd6-6f36-4bd4-8e96-7e1fec80d56f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:24.628539 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:24.629756 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:24.630163 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:24.630520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5 admin admin] PATH_INFO: `/v3/projects/ca090846b79e4879b9643ce0e1925a5f` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:24.632959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ca090846b79e4879b9643ce0e1925a5f)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:24.646250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:24.705986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ca090846b79e4879b9643ce0e1925a5f', 'request_id': 'req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:24.711771 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ca090846b79e4879b9643ce0e1925a5f', 'request_id': 'req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:24.720283 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ca090846b79e4879b9643ce0e1925a5f', 'request_id': 'req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:24.732070 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5 admin admin] The token cache is being invalidate because project ca090846b79e4879b9643ce0e1925a5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:15:24.732403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:24.733059 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:24.734710 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1286/2566] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:15:24 2022] DELETE /identity/v3/projects/ca090846b79e4879b9643ce0e1925a5f => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:24.738865 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88892e4-68b2-42f1-8068-8dd6477cacab service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:24.750057 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88892e4-68b2-42f1-8068-8dd6477cacab service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:24.817894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4fa8938-e99f-4bd6-9420-287a54b8eae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:24.818802 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa8938-e99f-4bd6-9420-287a54b8eae1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:24.819101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa8938-e99f-4bd6-9420-287a54b8eae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:24.820100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4fa8938-e99f-4bd6-9420-287a54b8eae1 admin admin] PATH_INFO: `/v3/users/5a4374e9425845b19b001c2271f2fe4f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:24.820833 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa8938-e99f-4bd6-9420-287a54b8eae1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a4374e9425845b19b001c2271f2fe4f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:24.822828 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4fa8938-e99f-4bd6-9420-287a54b8eae1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:24.895999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f4fa8938-e99f-4bd6-9420-287a54b8eae1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a4374e9425845b19b001c2271f2fe4f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:24.918710 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f4fa8938-e99f-4bd6-9420-287a54b8eae1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a4374e9425845b19b001c2271f2fe4f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:24.931419 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f4fa8938-e99f-4bd6-9420-287a54b8eae1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a4374e9425845b19b001c2271f2fe4f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:24.932177 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f4fa8938-e99f-4bd6-9420-287a54b8eae1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a4374e9425845b19b001c2271f2fe4f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:24.933824 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:24.950267 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f4fa8938-e99f-4bd6-9420-287a54b8eae1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a4374e9425845b19b001c2271f2fe4f'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:24.961298 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1281/2567] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:15:24 2022] DELETE /identity/v3/users/5a4374e9425845b19b001c2271f2fe4f => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:15:25.104549 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:25.106057 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:25.119073 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dafa1d2-995d-4d11-96f0-15525e1f8dc5 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:25.188307 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bc543-2b64-497d-a4a8-34bed17aad1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:25.191140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bc543-2b64-497d-a4a8-34bed17aad1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:25.191431 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bc543-2b64-497d-a4a8-34bed17aad1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:25.191728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e7bc543-2b64-497d-a4a8-34bed17aad1f admin admin] PATH_INFO: `/v3/users/4cc5e4c08912428388d36fcd2424a07d` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:25.193075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7bc543-2b64-497d-a4a8-34bed17aad1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=4cc5e4c08912428388d36fcd2424a07d)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:25.195211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e7bc543-2b64-497d-a4a8-34bed17aad1f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:25.263134 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1e7bc543-2b64-497d-a4a8-34bed17aad1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4cc5e4c08912428388d36fcd2424a07d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:25.263864 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1e7bc543-2b64-497d-a4a8-34bed17aad1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4cc5e4c08912428388d36fcd2424a07d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:25.271763 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:25.278558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1e7bc543-2b64-497d-a4a8-34bed17aad1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4cc5e4c08912428388d36fcd2424a07d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:25.287959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1e7bc543-2b64-497d-a4a8-34bed17aad1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4cc5e4c08912428388d36fcd2424a07d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:25.289046 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-1e7bc543-2b64-497d-a4a8-34bed17aad1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4cc5e4c08912428388d36fcd2424a07d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:25.315790 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1287/2568] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:15:25 2022] DELETE /identity/v3/users/4cc5e4c08912428388d36fcd2424a07d => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:25.320290 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde0694-da29-40a5-beca-82f7ce33e916 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:25.320896 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde0694-da29-40a5-beca-82f7ce33e916 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:25.321421 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bde0694-da29-40a5-beca-82f7ce33e916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:25.342776 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1bde0694-da29-40a5-beca-82f7ce33e916 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:25.344320 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1bde0694-da29-40a5-beca-82f7ce33e916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:25.429509 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1282/2569] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:25.443396 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bc543-2b64-497d-a4a8-34bed17aad1f admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:25.445275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-1e7bc543-2b64-497d-a4a8-34bed17aad1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:25.550314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e7bc543-2b64-497d-a4a8-34bed17aad1f admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:25.555005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132b1ea-195c-4d81-9ab4-8a88ceaa8f9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:25.555770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132b1ea-195c-4d81-9ab4-8a88ceaa8f9e service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:25.556101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132b1ea-195c-4d81-9ab4-8a88ceaa8f9e service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:25.556452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6132b1ea-195c-4d81-9ab4-8a88ceaa8f9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:25.560541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde0694-da29-40a5-beca-82f7ce33e916 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:25.563332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1bde0694-da29-40a5-beca-82f7ce33e916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:25.568082 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132b1ea-195c-4d81-9ab4-8a88ceaa8f9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:25.569838 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6132b1ea-195c-4d81-9ab4-8a88ceaa8f9e service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:25.582838 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bde0694-da29-40a5-beca-82f7ce33e916 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:25.586939 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1288/2570] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:25.598576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ba8ea-3ac5-49bc-9477-096072a16bf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:25.601192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ba8ea-3ac5-49bc-9477-096072a16bf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:25.602114 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ba8ea-3ac5-49bc-9477-096072a16bf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:25.602983 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073ba8ea-3ac5-49bc-9477-096072a16bf8 admin admin] PATH_INFO: `/v3/projects/7ce5cd4a15e8466784af0a4d93f4d0fd` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:25.604485 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ba8ea-3ac5-49bc-9477-096072a16bf8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7ce5cd4a15e8466784af0a4d93f4d0fd)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:25.607262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073ba8ea-3ac5-49bc-9477-096072a16bf8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:25.695945 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-073ba8ea-3ac5-49bc-9477-096072a16bf8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7ce5cd4a15e8466784af0a4d93f4d0fd', 'request_id': 'req-073ba8ea-3ac5-49bc-9477-096072a16bf8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:25.720003 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-073ba8ea-3ac5-49bc-9477-096072a16bf8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7ce5cd4a15e8466784af0a4d93f4d0fd', 'request_id': 'req-073ba8ea-3ac5-49bc-9477-096072a16bf8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:25.728613 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-073ba8ea-3ac5-49bc-9477-096072a16bf8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7ce5cd4a15e8466784af0a4d93f4d0fd', 'request_id': 'req-073ba8ea-3ac5-49bc-9477-096072a16bf8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:25.741635 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-073ba8ea-3ac5-49bc-9477-096072a16bf8 admin admin] The token cache is being invalidate because project 7ce5cd4a15e8466784af0a4d93f4d0fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:15:25.742294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-073ba8ea-3ac5-49bc-9477-096072a16bf8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:25.742610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-073ba8ea-3ac5-49bc-9477-096072a16bf8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:25.744628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1283/2571] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:15:25 2022] DELETE /identity/v3/projects/7ce5cd4a15e8466784af0a4d93f4d0fd => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:25.750075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132b1ea-195c-4d81-9ab4-8a88ceaa8f9e service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:25.769330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6132b1ea-195c-4d81-9ab4-8a88ceaa8f9e service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:25.848048 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f6582-1424-44ae-887f-b5b88d9a26e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:25.849172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f6582-1424-44ae-887f-b5b88d9a26e2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:25.849578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f6582-1424-44ae-887f-b5b88d9a26e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:25.849954 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5f6582-1424-44ae-887f-b5b88d9a26e2 admin admin] PATH_INFO: `/v3/users/b217a1c8d81046beb00b11fdd37af035` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:25.851355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f6582-1424-44ae-887f-b5b88d9a26e2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b217a1c8d81046beb00b11fdd37af035)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:25.854143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5f6582-1424-44ae-887f-b5b88d9a26e2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:25.915974 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5b5f6582-1424-44ae-887f-b5b88d9a26e2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b217a1c8d81046beb00b11fdd37af035'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:25.916692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5b5f6582-1424-44ae-887f-b5b88d9a26e2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b217a1c8d81046beb00b11fdd37af035'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:25.931678 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ba8ea-3ac5-49bc-9477-096072a16bf8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:25.933159 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-073ba8ea-3ac5-49bc-9477-096072a16bf8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:25.936296 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5b5f6582-1424-44ae-887f-b5b88d9a26e2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b217a1c8d81046beb00b11fdd37af035'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:25.949990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5b5f6582-1424-44ae-887f-b5b88d9a26e2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b217a1c8d81046beb00b11fdd37af035'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:25.951033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-5b5f6582-1424-44ae-887f-b5b88d9a26e2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b217a1c8d81046beb00b11fdd37af035'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:25.952208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073ba8ea-3ac5-49bc-9477-096072a16bf8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:25.958294 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:25.959334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:25.959775 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:25.961262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4 admin admin] PATH_INFO: `/v3/projects/3796dc34098747fabe0ae777a41a0551` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:25.962374 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3796dc34098747fabe0ae777a41a0551)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:25.964574 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:25.971683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:25.986490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1289/2572] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:15:25 2022] DELETE /identity/v3/users/b217a1c8d81046beb00b11fdd37af035 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:26.029276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3796dc34098747fabe0ae777a41a0551', 'request_id': 'req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:26.040289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3796dc34098747fabe0ae777a41a0551', 'request_id': 'req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:26.052300 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3796dc34098747fabe0ae777a41a0551', 'request_id': 'req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:26.058811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:26.069367 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4 admin admin] The token cache is being invalidate because project 3796dc34098747fabe0ae777a41a0551 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:15:26.069900 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:26.070693 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:26.075786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1284/2573] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:15:25 2022] DELETE /identity/v3/projects/3796dc34098747fabe0ae777a41a0551 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:15:26.077033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f6582-1424-44ae-887f-b5b88d9a26e2 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:26.078792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5b5f6582-1424-44ae-887f-b5b88d9a26e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:26.197273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5f6582-1424-44ae-887f-b5b88d9a26e2 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:26.291671 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13515e46-c526-4d5f-bba7-c60cdbd9acf0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:26.291671 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13515e46-c526-4d5f-bba7-c60cdbd9acf0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:26.297736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13515e46-c526-4d5f-bba7-c60cdbd9acf0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:26.298030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13515e46-c526-4d5f-bba7-c60cdbd9acf0 admin admin] PATH_INFO: `/v3/users/3d665dee2c3e4650b1807a1eeab8f57a` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:26.307589 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13515e46-c526-4d5f-bba7-c60cdbd9acf0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d665dee2c3e4650b1807a1eeab8f57a)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:26.310003 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13515e46-c526-4d5f-bba7-c60cdbd9acf0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:26.325344 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:26.327262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:26.348155 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11238c5a-9a3f-4a90-b05e-8cc8ecb74de4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:26.353906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff7ffc8-2f59-4baa-82a4-2e39ea24215b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:26.354996 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7ffc8-2f59-4baa-82a4-2e39ea24215b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:26.355383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7ffc8-2f59-4baa-82a4-2e39ea24215b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:26.355737 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff7ffc8-2f59-4baa-82a4-2e39ea24215b admin admin] PATH_INFO: `/v3/projects/11ccf6bcfaa8411aa2ffb6cb59c90bc6` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:26.357106 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7ffc8-2f59-4baa-82a4-2e39ea24215b admin admin] RBAC: Authorizing `identity:delete_project(project_id=11ccf6bcfaa8411aa2ffb6cb59c90bc6)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:26.359886 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff7ffc8-2f59-4baa-82a4-2e39ea24215b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:26.406264 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-13515e46-c526-4d5f-bba7-c60cdbd9acf0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d665dee2c3e4650b1807a1eeab8f57a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:26.407026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-13515e46-c526-4d5f-bba7-c60cdbd9acf0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d665dee2c3e4650b1807a1eeab8f57a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:26.420326 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-13515e46-c526-4d5f-bba7-c60cdbd9acf0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d665dee2c3e4650b1807a1eeab8f57a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:26.422648 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7ff7ffc8-2f59-4baa-82a4-2e39ea24215b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11ccf6bcfaa8411aa2ffb6cb59c90bc6', 'request_id': 'req-7ff7ffc8-2f59-4baa-82a4-2e39ea24215b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:26.432755 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-13515e46-c526-4d5f-bba7-c60cdbd9acf0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d665dee2c3e4650b1807a1eeab8f57a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:26.433842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-13515e46-c526-4d5f-bba7-c60cdbd9acf0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d665dee2c3e4650b1807a1eeab8f57a'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:26.437921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7ff7ffc8-2f59-4baa-82a4-2e39ea24215b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11ccf6bcfaa8411aa2ffb6cb59c90bc6', 'request_id': 'req-7ff7ffc8-2f59-4baa-82a4-2e39ea24215b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:26.449219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7ff7ffc8-2f59-4baa-82a4-2e39ea24215b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11ccf6bcfaa8411aa2ffb6cb59c90bc6', 'request_id': 'req-7ff7ffc8-2f59-4baa-82a4-2e39ea24215b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:26.456026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1290/2574] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:15:26 2022] DELETE /identity/v3/users/3d665dee2c3e4650b1807a1eeab8f57a => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:26.461556 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7ff7ffc8-2f59-4baa-82a4-2e39ea24215b admin admin] The token cache is being invalidate because project 11ccf6bcfaa8411aa2ffb6cb59c90bc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:15:26.461882 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7ff7ffc8-2f59-4baa-82a4-2e39ea24215b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:26.462592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-7ff7ffc8-2f59-4baa-82a4-2e39ea24215b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:26.464492 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1285/2575] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:15:26 2022] DELETE /identity/v3/projects/11ccf6bcfaa8411aa2ffb6cb59c90bc6 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:26.473533 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88359cd-f9dc-4f8c-b986-221bd9fdaf74 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:26.473818 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88359cd-f9dc-4f8c-b986-221bd9fdaf74 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:26.474273 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88359cd-f9dc-4f8c-b986-221bd9fdaf74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:26.498160 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c88359cd-f9dc-4f8c-b986-221bd9fdaf74 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:26.499481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c88359cd-f9dc-4f8c-b986-221bd9fdaf74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:26.524957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:26.591452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1291/2576] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:15:26.593621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2bda5-fd35-4972-8625-2ca3e1141659 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:26.593855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2bda5-fd35-4972-8625-2ca3e1141659 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:26.594108 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd2bda5-fd35-4972-8625-2ca3e1141659 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:26.613623 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1cd2bda5-fd35-4972-8625-2ca3e1141659 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:26.614953 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1cd2bda5-fd35-4972-8625-2ca3e1141659 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:26.624390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1286/2577] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:26.629197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88359cd-f9dc-4f8c-b986-221bd9fdaf74 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:26.641030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88359cd-f9dc-4f8c-b986-221bd9fdaf74 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:26.646840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017bcf4-6291-48ad-b1d9-6e1882436085 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:26.647643 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017bcf4-6291-48ad-b1d9-6e1882436085 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:26.648069 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017bcf4-6291-48ad-b1d9-6e1882436085 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:26.648558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2017bcf4-6291-48ad-b1d9-6e1882436085 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:26.650007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017bcf4-6291-48ad-b1d9-6e1882436085 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:15:26.650489 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017bcf4-6291-48ad-b1d9-6e1882436085 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:26.652486 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2017bcf4-6291-48ad-b1d9-6e1882436085 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:26.671999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1292/2578] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:15:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:15:26.675542 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2bda5-fd35-4972-8625-2ca3e1141659 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:26.694706 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd2bda5-fd35-4972-8625-2ca3e1141659 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:26.700322 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091650c-132c-4660-90eb-52538665b32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:26.701512 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091650c-132c-4660-90eb-52538665b32d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:26.701904 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091650c-132c-4660-90eb-52538665b32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:26.702263 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e091650c-132c-4660-90eb-52538665b32d admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:26.703187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091650c-132c-4660-90eb-52538665b32d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:26.705385 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e091650c-132c-4660-90eb-52538665b32d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:26.741066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1287/2579] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:15:26 2022] POST /identity/v3/projects => generated 407 bytes in 66 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:26.745137 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017bcf4-6291-48ad-b1d9-6e1882436085 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:26.756417 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2017bcf4-6291-48ad-b1d9-6e1882436085 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:26.821903 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1d7da-101d-4672-ac75-47f3c0eeb242 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:26.822691 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1d7da-101d-4672-ac75-47f3c0eeb242 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:26.823051 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1d7da-101d-4672-ac75-47f3c0eeb242 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:26.823933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a1d7da-101d-4672-ac75-47f3c0eeb242 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:26.824482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1d7da-101d-4672-ac75-47f3c0eeb242 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:26.827199 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a1d7da-101d-4672-ac75-47f3c0eeb242 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:26.887749 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1293/2580] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:15:26 2022] POST /identity/v3/users => generated 304 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:26.891707 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091650c-132c-4660-90eb-52538665b32d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:26.906035 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e091650c-132c-4660-90eb-52538665b32d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:26.912719 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611d63f-03bc-4b89-bb4c-16d40774e373 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:26.913412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611d63f-03bc-4b89-bb4c-16d40774e373 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:26.913705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611d63f-03bc-4b89-bb4c-16d40774e373 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:26.913989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5611d63f-03bc-4b89-bb4c-16d40774e373 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:26.914729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611d63f-03bc-4b89-bb4c-16d40774e373 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:26.916596 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5611d63f-03bc-4b89-bb4c-16d40774e373 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:26.931878 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1288/2581] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:26.934340 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1d7da-101d-4672-ac75-47f3c0eeb242 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:26.946363 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a1d7da-101d-4672-ac75-47f3c0eeb242 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:26.951244 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d61ce-68f4-4e29-bc51-7f0043f6db5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:26.952031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d61ce-68f4-4e29-bc51-7f0043f6db5c admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:26.952403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d61ce-68f4-4e29-bc51-7f0043f6db5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:26.952687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8d61ce-68f4-4e29-bc51-7f0043f6db5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:26.953459 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d61ce-68f4-4e29-bc51-7f0043f6db5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:26.955863 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8d61ce-68f4-4e29-bc51-7f0043f6db5c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:26.969731 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1294/2582] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:26 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:26.970904 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611d63f-03bc-4b89-bb4c-16d40774e373 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:26.981879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5611d63f-03bc-4b89-bb4c-16d40774e373 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:26.986908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-652ae1a8-1ff6-470e-b134-aa2ede9a3d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:26.988104 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ae1a8-1ff6-470e-b134-aa2ede9a3d2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:26.988398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ae1a8-1ff6-470e-b134-aa2ede9a3d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:26.988717 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-652ae1a8-1ff6-470e-b134-aa2ede9a3d2e admin admin] PATH_INFO: `/v3/projects/47bc58f24f25465fb316bb127af08cbc/users/2b02c19481564b53b14ca8448140f206/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.000664 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d61ce-68f4-4e29-bc51-7f0043f6db5c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:27.003144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-cc8d61ce-68f4-4e29-bc51-7f0043f6db5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:27.008514 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ae1a8-1ff6-470e-b134-aa2ede9a3d2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=47bc58f24f25465fb316bb127af08cbc, user_id=2b02c19481564b53b14ca8448140f206, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:27.011254 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-652ae1a8-1ff6-470e-b134-aa2ede9a3d2e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:27.018808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8d61ce-68f4-4e29-bc51-7f0043f6db5c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:27.026742 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92896ae6-72c8-4500-969a-0a28db0de881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:27.027735 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92896ae6-72c8-4500-969a-0a28db0de881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.028229 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92896ae6-72c8-4500-969a-0a28db0de881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.038782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1289/2583] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:26 2022] PUT /identity/v3/projects/47bc58f24f25465fb316bb127af08cbc/users/2b02c19481564b53b14ca8448140f206/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:27.039455 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92896ae6-72c8-4500-969a-0a28db0de881 admin admin] PATH_INFO: `/v3/projects/01be1da020174168a8bef1a3f1a433e3` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.039455 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92896ae6-72c8-4500-969a-0a28db0de881 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01be1da020174168a8bef1a3f1a433e3)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:27.039455 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92896ae6-72c8-4500-969a-0a28db0de881 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:27.042517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7566e8-aa8a-45c1-8a08-85a1008ab056 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.042819 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7566e8-aa8a-45c1-8a08-85a1008ab056 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.043109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb7566e8-aa8a-45c1-8a08-85a1008ab056 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.074576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bb7566e8-aa8a-45c1-8a08-85a1008ab056 None None] MFA Rules not processed for user `2b02c19481564b53b14ca8448140f206`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:27.077084 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bb7566e8-aa8a-45c1-8a08-85a1008ab056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:27.098681 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-92896ae6-72c8-4500-969a-0a28db0de881 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01be1da020174168a8bef1a3f1a433e3', 'request_id': 'req-92896ae6-72c8-4500-969a-0a28db0de881'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:27.105163 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-92896ae6-72c8-4500-969a-0a28db0de881 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01be1da020174168a8bef1a3f1a433e3', 'request_id': 'req-92896ae6-72c8-4500-969a-0a28db0de881'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:27.113784 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-92896ae6-72c8-4500-969a-0a28db0de881 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01be1da020174168a8bef1a3f1a433e3', 'request_id': 'req-92896ae6-72c8-4500-969a-0a28db0de881'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:27.128208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-92896ae6-72c8-4500-969a-0a28db0de881 admin admin] The token cache is being invalidate because project 01be1da020174168a8bef1a3f1a433e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:15:27.128511 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-92896ae6-72c8-4500-969a-0a28db0de881 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:27.129199 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-92896ae6-72c8-4500-969a-0a28db0de881 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:27.130830 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1295/2584] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:15:26 2022] DELETE /identity/v3/projects/01be1da020174168a8bef1a3f1a433e3 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:15:27.135909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596afc7-ea29-4ee1-9514-33d406f1b21f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.135909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596afc7-ea29-4ee1-9514-33d406f1b21f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.136267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9596afc7-ea29-4ee1-9514-33d406f1b21f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.155716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1290/2585] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:27 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:27.157634 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-9596afc7-ea29-4ee1-9514-33d406f1b21f None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:27.158894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9596afc7-ea29-4ee1-9514-33d406f1b21f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:27.160512 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7566e8-aa8a-45c1-8a08-85a1008ab056 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:27.161828 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bb7566e8-aa8a-45c1-8a08-85a1008ab056 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:27.165605 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:27.167880 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:27.231741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1296/2586] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:15:27.234451 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb7566e8-aa8a-45c1-8a08-85a1008ab056 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:27.239215 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01116d63-da1d-4220-9415-36cafcfa8a2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:27.239398 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00c521-c099-4cf1-9ebb-30903b0ad855 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.239648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00c521-c099-4cf1-9ebb-30903b0ad855 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.239966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b00c521-c099-4cf1-9ebb-30903b0ad855 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.240143 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01116d63-da1d-4220-9415-36cafcfa8a2d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.240307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01116d63-da1d-4220-9415-36cafcfa8a2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.240625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01116d63-da1d-4220-9415-36cafcfa8a2d admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.241387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01116d63-da1d-4220-9415-36cafcfa8a2d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:27.243362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01116d63-da1d-4220-9415-36cafcfa8a2d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:27.259255 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-9b00c521-c099-4cf1-9ebb-30903b0ad855 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:27.260499 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9b00c521-c099-4cf1-9ebb-30903b0ad855 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:27.268710 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1297/2587] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:15:27.273198 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1291/2588] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:15:27 2022] POST /identity/v3/projects => generated 409 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:15:27.273363 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00c521-c099-4cf1-9ebb-30903b0ad855 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:27.274119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01116d63-da1d-4220-9415-36cafcfa8a2d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:27.282887 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b00c521-c099-4cf1-9ebb-30903b0ad855 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:27.338883 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01116d63-da1d-4220-9415-36cafcfa8a2d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:27.343726 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad3438-d202-4f51-be38-2d7ab1a53a8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:27.344436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad3438-d202-4f51-be38-2d7ab1a53a8b admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.344721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad3438-d202-4f51-be38-2d7ab1a53a8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.345004 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55ad3438-d202-4f51-be38-2d7ab1a53a8b admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.345626 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d472aa4-66a7-4b74-966b-0b95ac8c2e10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:27.345749 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad3438-d202-4f51-be38-2d7ab1a53a8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:27.346315 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d472aa4-66a7-4b74-966b-0b95ac8c2e10 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.346615 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d472aa4-66a7-4b74-966b-0b95ac8c2e10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.346885 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d472aa4-66a7-4b74-966b-0b95ac8c2e10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.347627 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55ad3438-d202-4f51-be38-2d7ab1a53a8b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:27.347727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d472aa4-66a7-4b74-966b-0b95ac8c2e10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:15:27.348150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d472aa4-66a7-4b74-966b-0b95ac8c2e10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:27.349831 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d472aa4-66a7-4b74-966b-0b95ac8c2e10 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:27.361969 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1298/2589] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:15:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:15:27.366488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d472aa4-66a7-4b74-966b-0b95ac8c2e10 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:27.374313 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1292/2590] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:15:27 2022] POST /identity/v3/users => generated 311 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:15:27.376469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d472aa4-66a7-4b74-966b-0b95ac8c2e10 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:27.378589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad3438-d202-4f51-be38-2d7ab1a53a8b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:27.381432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bd474-b8e9-4109-8f34-54e7ad2eb87b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:27.382095 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bd474-b8e9-4109-8f34-54e7ad2eb87b admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.382384 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bd474-b8e9-4109-8f34-54e7ad2eb87b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.382671 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-259bd474-b8e9-4109-8f34-54e7ad2eb87b admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.383362 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bd474-b8e9-4109-8f34-54e7ad2eb87b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:27.385383 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-259bd474-b8e9-4109-8f34-54e7ad2eb87b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:27.388658 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55ad3438-d202-4f51-be38-2d7ab1a53a8b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:27.393561 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8057d-bb14-41f4-b521-351dc4a2e3c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:27.394234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8057d-bb14-41f4-b521-351dc4a2e3c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.394525 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8057d-bb14-41f4-b521-351dc4a2e3c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.394818 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b8057d-bb14-41f4-b521-351dc4a2e3c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.395549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8057d-bb14-41f4-b521-351dc4a2e3c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:27.397205 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b8057d-bb14-41f4-b521-351dc4a2e3c8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:27.411354 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1293/2591] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:15:27.412414 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8057d-bb14-41f4-b521-351dc4a2e3c8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:27.419559 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1299/2592] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:15:27 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:15:27.420592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bd474-b8e9-4109-8f34-54e7ad2eb87b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:27.422119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b8057d-bb14-41f4-b521-351dc4a2e3c8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:27.480275 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22146fcb-eabb-4d50-a65d-beb4c1e39801 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:27.481186 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22146fcb-eabb-4d50-a65d-beb4c1e39801 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.481506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22146fcb-eabb-4d50-a65d-beb4c1e39801 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.481791 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22146fcb-eabb-4d50-a65d-beb4c1e39801 admin admin] PATH_INFO: `/v3/projects/6eba0ee08ef345a29dd52abf8d563095/users/ccff097ab42f42b490ae7defcfa43706/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.482218 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-259bd474-b8e9-4109-8f34-54e7ad2eb87b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:27.486897 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76cf18-76a8-4a93-af9b-e05a241afc6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:27.487569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76cf18-76a8-4a93-af9b-e05a241afc6b admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.487892 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76cf18-76a8-4a93-af9b-e05a241afc6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.488174 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f76cf18-76a8-4a93-af9b-e05a241afc6b admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.488860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f76cf18-76a8-4a93-af9b-e05a241afc6b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:27.490786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f76cf18-76a8-4a93-af9b-e05a241afc6b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:27.494648 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:27.498514 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22146fcb-eabb-4d50-a65d-beb4c1e39801 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eba0ee08ef345a29dd52abf8d563095, user_id=ccff097ab42f42b490ae7defcfa43706, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:27.502497 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22146fcb-eabb-4d50-a65d-beb4c1e39801 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:27.512927 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1300/2593] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:15:27 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:15:27.516308 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1294/2594] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:27 2022] PUT /identity/v3/projects/6eba0ee08ef345a29dd52abf8d563095/users/ccff097ab42f42b490ae7defcfa43706/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:15:27.517060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76cf18-76a8-4a93-af9b-e05a241afc6b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:27.519868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22146fcb-eabb-4d50-a65d-beb4c1e39801 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:27.526659 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f76cf18-76a8-4a93-af9b-e05a241afc6b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:27.530537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:27.582434 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22146fcb-eabb-4d50-a65d-beb4c1e39801 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:27.584094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4b9f6-e557-4848-b35b-6c356b00fdf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:27.584783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4b9f6-e557-4848-b35b-6c356b00fdf4 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.585071 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4b9f6-e557-4848-b35b-6c356b00fdf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.585428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10c4b9f6-e557-4848-b35b-6c356b00fdf4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.586172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4b9f6-e557-4848-b35b-6c356b00fdf4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:27.587383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195fc25-cf5c-4bcf-97f4-ee0eaee0a80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:27.587792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10c4b9f6-e557-4848-b35b-6c356b00fdf4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:27.588093 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195fc25-cf5c-4bcf-97f4-ee0eaee0a80c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.588386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195fc25-cf5c-4bcf-97f4-ee0eaee0a80c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.588669 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4195fc25-cf5c-4bcf-97f4-ee0eaee0a80c admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.589409 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195fc25-cf5c-4bcf-97f4-ee0eaee0a80c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:27.591113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4195fc25-cf5c-4bcf-97f4-ee0eaee0a80c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:27.597849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1301/2595] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:27.601826 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4b9f6-e557-4848-b35b-6c356b00fdf4 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:27.605820 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1295/2596] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:15:27.606789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195fc25-cf5c-4bcf-97f4-ee0eaee0a80c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:27.612167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10c4b9f6-e557-4848-b35b-6c356b00fdf4 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:27.616439 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4195fc25-cf5c-4bcf-97f4-ee0eaee0a80c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:27.616684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3351d4-04c4-4fdb-8e24-c9f2164df6c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:27.617354 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3351d4-04c4-4fdb-8e24-c9f2164df6c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.617640 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3351d4-04c4-4fdb-8e24-c9f2164df6c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.617919 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3351d4-04c4-4fdb-8e24-c9f2164df6c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.618639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3351d4-04c4-4fdb-8e24-c9f2164df6c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:27.620293 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3351d4-04c4-4fdb-8e24-c9f2164df6c7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:27.621326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90907aec-9cdc-4bb0-8279-c08c3d682730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:27.622262 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90907aec-9cdc-4bb0-8279-c08c3d682730 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.622552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90907aec-9cdc-4bb0-8279-c08c3d682730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.622837 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90907aec-9cdc-4bb0-8279-c08c3d682730 admin admin] PATH_INFO: `/v3/projects/6eba0ee08ef345a29dd52abf8d563095/users/ccff097ab42f42b490ae7defcfa43706/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.625069 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90907aec-9cdc-4bb0-8279-c08c3d682730 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6eba0ee08ef345a29dd52abf8d563095, user_id=ccff097ab42f42b490ae7defcfa43706, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:27.628360 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90907aec-9cdc-4bb0-8279-c08c3d682730 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:27.630928 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1302/2597] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:27 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:27.636362 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3351d4-04c4-4fdb-8e24-c9f2164df6c7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:27.640267 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1296/2598] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:27 2022] PUT /identity/v3/projects/6eba0ee08ef345a29dd52abf8d563095/users/ccff097ab42f42b490ae7defcfa43706/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:27.646270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3351d4-04c4-4fdb-8e24-c9f2164df6c7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:27.652246 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafab20-0b50-41d9-9752-0aebb725ee28 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.652537 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafab20-0b50-41d9-9752-0aebb725ee28 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.652817 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fafab20-0b50-41d9-9752-0aebb725ee28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.679539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-6fafab20-0b50-41d9-9752-0aebb725ee28 None None] MFA Rules not processed for user `ccff097ab42f42b490ae7defcfa43706`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:27.680827 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6fafab20-0b50-41d9-9752-0aebb725ee28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:27.707264 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199d7864-8b71-449b-a4b1-78951270cd60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:27.708247 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d7864-8b71-449b-a4b1-78951270cd60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.708530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d7864-8b71-449b-a4b1-78951270cd60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.708808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199d7864-8b71-449b-a4b1-78951270cd60 admin admin] PATH_INFO: `/v3/projects/b0f06a5d4c67452dbd4c860175726d4f/users/81751664267043eab0c71bb0b2e95692/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.723298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d7864-8b71-449b-a4b1-78951270cd60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b0f06a5d4c67452dbd4c860175726d4f, user_id=81751664267043eab0c71bb0b2e95692, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:27.726004 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199d7864-8b71-449b-a4b1-78951270cd60 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:27.740680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1303/2599] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:27 2022] PUT /identity/v3/projects/b0f06a5d4c67452dbd4c860175726d4f/users/81751664267043eab0c71bb0b2e95692/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:15:27.749535 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1297/2600] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:27 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:15:27.752764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb5a16-9e34-4cee-9483-4ab2c67b837c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.753060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb5a16-9e34-4cee-9483-4ab2c67b837c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.753375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb5a16-9e34-4cee-9483-4ab2c67b837c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.761502 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fafab20-0b50-41d9-9752-0aebb725ee28 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:27.762931 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6fafab20-0b50-41d9-9752-0aebb725ee28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:27.780777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-9abb5a16-9e34-4cee-9483-4ab2c67b837c None None] MFA Rules not processed for user `81751664267043eab0c71bb0b2e95692`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:27.781987 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9abb5a16-9e34-4cee-9483-4ab2c67b837c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:27.820443 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fafab20-0b50-41d9-9752-0aebb725ee28 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:27.831917 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:27.845729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1304/2601] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:27 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:27.849946 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb5a16-9e34-4cee-9483-4ab2c67b837c None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:27.855437 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:27.881261 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae84ac0-a039-47db-b01f-8f1efc523898 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:27.882060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae84ac0-a039-47db-b01f-8f1efc523898 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.882359 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae84ac0-a039-47db-b01f-8f1efc523898 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.882647 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dae84ac0-a039-47db-b01f-8f1efc523898 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.916452 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb5a16-9e34-4cee-9483-4ab2c67b837c None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:27.922314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd45560-fdde-41fb-b157-582102893fd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:27.922314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd45560-fdde-41fb-b157-582102893fd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:27.922314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd45560-fdde-41fb-b157-582102893fd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:27.922314 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd45560-fdde-41fb-b157-582102893fd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:27.922986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd45560-fdde-41fb-b157-582102893fd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:27.924961 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd45560-fdde-41fb-b157-582102893fd0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:27.945559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae84ac0-a039-47db-b01f-8f1efc523898 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:27.947131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dae84ac0-a039-47db-b01f-8f1efc523898 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:27.958309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1298/2603] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:27 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:27.958453 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1305/2603] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:15:27 2022] POST /identity/v3/projects => generated 377 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:15:27.968934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae84ac0-a039-47db-b01f-8f1efc523898 service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:27.984960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dae84ac0-a039-47db-b01f-8f1efc523898 service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:28.056095 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb05dc-3c7b-4d81-8b69-15a87ec43778 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:28.056826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb05dc-3c7b-4d81-8b69-15a87ec43778 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:28.057296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb05dc-3c7b-4d81-8b69-15a87ec43778 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:28.057585 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bbb05dc-3c7b-4d81-8b69-15a87ec43778 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:28.058325 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb05dc-3c7b-4d81-8b69-15a87ec43778 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:28.060375 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bbb05dc-3c7b-4d81-8b69-15a87ec43778 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:28.082781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1299/2604] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:15:27 2022] POST /identity/v3/users => generated 295 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:28.087763 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd45560-fdde-41fb-b157-582102893fd0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:28.098467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd45560-fdde-41fb-b157-582102893fd0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:28.103205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42570c36-337f-4a60-b2eb-c7186cf07c91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:28.103923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42570c36-337f-4a60-b2eb-c7186cf07c91 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:28.104207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42570c36-337f-4a60-b2eb-c7186cf07c91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:28.104494 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42570c36-337f-4a60-b2eb-c7186cf07c91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:28.105223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42570c36-337f-4a60-b2eb-c7186cf07c91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:28.106946 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42570c36-337f-4a60-b2eb-c7186cf07c91 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:28.117698 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1306/2605] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:15:28.122104 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb05dc-3c7b-4d81-8b69-15a87ec43778 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:28.137009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bbb05dc-3c7b-4d81-8b69-15a87ec43778 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:28.138229 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42570c36-337f-4a60-b2eb-c7186cf07c91 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:28.141715 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af4d89-dc41-41a4-9eff-8221f4ba7def admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:28.142591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af4d89-dc41-41a4-9eff-8221f4ba7def admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:28.142882 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af4d89-dc41-41a4-9eff-8221f4ba7def admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:28.143162 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15af4d89-dc41-41a4-9eff-8221f4ba7def admin admin] PATH_INFO: `/v3/projects/017fe8e6b6a14532938922a038574b0b/users/f3ead1a34f94452197fd51401b7f2b64/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:28.160395 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af4d89-dc41-41a4-9eff-8221f4ba7def admin admin] RBAC: Authorizing `identity:create_grant(project_id=017fe8e6b6a14532938922a038574b0b, user_id=f3ead1a34f94452197fd51401b7f2b64, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:28.163081 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15af4d89-dc41-41a4-9eff-8221f4ba7def admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:28.178021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1300/2606] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:28 2022] PUT /identity/v3/projects/017fe8e6b6a14532938922a038574b0b/users/f3ead1a34f94452197fd51401b7f2b64/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:28.182179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af4d89-dc41-41a4-9eff-8221f4ba7def admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:28.195499 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42570c36-337f-4a60-b2eb-c7186cf07c91 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:28.216058 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:28.245442 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146dec7-1d04-45a3-8fa1-dfbd18f7e7a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:28.246111 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146dec7-1d04-45a3-8fa1-dfbd18f7e7a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:28.246401 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146dec7-1d04-45a3-8fa1-dfbd18f7e7a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:28.246683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9146dec7-1d04-45a3-8fa1-dfbd18f7e7a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:28.247009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15af4d89-dc41-41a4-9eff-8221f4ba7def admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:28.251584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959bd0e-7dd7-49b8-b668-389df2a77bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:28.252672 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959bd0e-7dd7-49b8-b668-389df2a77bac admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:28.252967 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959bd0e-7dd7-49b8-b668-389df2a77bac admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:28.253250 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1959bd0e-7dd7-49b8-b668-389df2a77bac admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:28.253969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959bd0e-7dd7-49b8-b668-389df2a77bac admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:28.255592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1959bd0e-7dd7-49b8-b668-389df2a77bac admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:28.266143 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1301/2607] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:15:28.270231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959bd0e-7dd7-49b8-b668-389df2a77bac admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:28.279971 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1959bd0e-7dd7-49b8-b668-389df2a77bac admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:28.284656 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1330559-a64a-4e80-a1fa-43dc0f6eed19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:28.285533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1330559-a64a-4e80-a1fa-43dc0f6eed19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:28.285822 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1330559-a64a-4e80-a1fa-43dc0f6eed19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:28.286105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1330559-a64a-4e80-a1fa-43dc0f6eed19 admin admin] PATH_INFO: `/v3/projects/017fe8e6b6a14532938922a038574b0b/users/f3ead1a34f94452197fd51401b7f2b64/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:28.288233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1330559-a64a-4e80-a1fa-43dc0f6eed19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=017fe8e6b6a14532938922a038574b0b, user_id=f3ead1a34f94452197fd51401b7f2b64, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:28.290864 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1330559-a64a-4e80-a1fa-43dc0f6eed19 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:28.299902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:28.303783 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146dec7-1d04-45a3-8fa1-dfbd18f7e7a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:28.304229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1302/2608] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:28 2022] PUT /identity/v3/projects/017fe8e6b6a14532938922a038574b0b/users/f3ead1a34f94452197fd51401b7f2b64/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:15:28.305753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9146dec7-1d04-45a3-8fa1-dfbd18f7e7a6 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:28.314396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842be18-aa83-4de9-ba89-7d9b3c8562f4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:28.314720 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842be18-aa83-4de9-ba89-7d9b3c8562f4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:28.315013 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1842be18-aa83-4de9-ba89-7d9b3c8562f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:28.316964 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1307/2609] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:28 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 179 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:15:28.345751 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1842be18-aa83-4de9-ba89-7d9b3c8562f4 None None] MFA Rules not processed for user `f3ead1a34f94452197fd51401b7f2b64`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:28.347029 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1842be18-aa83-4de9-ba89-7d9b3c8562f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:28.415969 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1303/2610] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:28 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:15:28.428704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146dec7-1d04-45a3-8fa1-dfbd18f7e7a6 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:28.430150 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9146dec7-1d04-45a3-8fa1-dfbd18f7e7a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:28.441514 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9146dec7-1d04-45a3-8fa1-dfbd18f7e7a6 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:28.494002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267ae4d-6d19-4b6d-b256-8f5dc2240798 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:28.494674 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267ae4d-6d19-4b6d-b256-8f5dc2240798 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:28.494963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267ae4d-6d19-4b6d-b256-8f5dc2240798 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:28.495256 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b267ae4d-6d19-4b6d-b256-8f5dc2240798 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:28.568138 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267ae4d-6d19-4b6d-b256-8f5dc2240798 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:28.569608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b267ae4d-6d19-4b6d-b256-8f5dc2240798 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:28.581564 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1308/2611] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:38.413001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=54,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:38.577167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=53 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:39.428046 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842be18-aa83-4de9-ba89-7d9b3c8562f4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:39.438425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1842be18-aa83-4de9-ba89-7d9b3c8562f4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:39.442559 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768719a8-0d75-4ab9-bbda-7b30af6b9fe2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:39.443200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768719a8-0d75-4ab9-bbda-7b30af6b9fe2 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:39.443518 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768719a8-0d75-4ab9-bbda-7b30af6b9fe2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:39.443796 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768719a8-0d75-4ab9-bbda-7b30af6b9fe2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:39.455687 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768719a8-0d75-4ab9-bbda-7b30af6b9fe2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:39.457184 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-768719a8-0d75-4ab9-bbda-7b30af6b9fe2 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:39.466897 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1304/2612] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:39 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:40.903441 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267ae4d-6d19-4b6d-b256-8f5dc2240798 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:40.974197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b267ae4d-6d19-4b6d-b256-8f5dc2240798 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:40.978846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c8710-a399-48b8-802a-e38867f33dd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:40.979769 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c8710-a399-48b8-802a-e38867f33dd5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:40.980092 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c8710-a399-48b8-802a-e38867f33dd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:40.980372 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-350c8710-a399-48b8-802a-e38867f33dd5 admin admin] PATH_INFO: `/v3/users/2b02c19481564b53b14ca8448140f206` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:40.981641 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c8710-a399-48b8-802a-e38867f33dd5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b02c19481564b53b14ca8448140f206)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:40.983638 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-350c8710-a399-48b8-802a-e38867f33dd5 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:41.036328 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-350c8710-a399-48b8-802a-e38867f33dd5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b02c19481564b53b14ca8448140f206'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:41.037050 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-350c8710-a399-48b8-802a-e38867f33dd5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b02c19481564b53b14ca8448140f206'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:41.050878 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-350c8710-a399-48b8-802a-e38867f33dd5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b02c19481564b53b14ca8448140f206'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:41.060690 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-350c8710-a399-48b8-802a-e38867f33dd5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b02c19481564b53b14ca8448140f206'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:41.061348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-350c8710-a399-48b8-802a-e38867f33dd5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b02c19481564b53b14ca8448140f206'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:41.081231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1309/2613] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:15:40 2022] DELETE /identity/v3/users/2b02c19481564b53b14ca8448140f206 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:41.086035 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad930b-bc03-40c2-8bc9-12f01037f204 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:41.086623 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad930b-bc03-40c2-8bc9-12f01037f204 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:41.086623 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fad930b-bc03-40c2-8bc9-12f01037f204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:41.105623 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1fad930b-bc03-40c2-8bc9-12f01037f204 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:41.106837 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1fad930b-bc03-40c2-8bc9-12f01037f204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:41.165903 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1305/2614] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:15:41.176056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c8710-a399-48b8-802a-e38867f33dd5 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:41.177880 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-350c8710-a399-48b8-802a-e38867f33dd5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:41.238653 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-350c8710-a399-48b8-802a-e38867f33dd5 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:41.242492 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b9d5b-928b-4352-8fc3-09c73674d5a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:41.243145 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b9d5b-928b-4352-8fc3-09c73674d5a2 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:41.243425 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b9d5b-928b-4352-8fc3-09c73674d5a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:41.243704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7b9d5b-928b-4352-8fc3-09c73674d5a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:41.253665 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b9d5b-928b-4352-8fc3-09c73674d5a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:41.255149 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7b9d5b-928b-4352-8fc3-09c73674d5a2 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:41.264913 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1310/2615] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:41.562798 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad930b-bc03-40c2-8bc9-12f01037f204 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:41.564795 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1fad930b-bc03-40c2-8bc9-12f01037f204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:41.577481 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fad930b-bc03-40c2-8bc9-12f01037f204 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:41.583127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:41.584075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:41.584347 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:41.584621 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2 admin admin] PATH_INFO: `/v3/projects/47bc58f24f25465fb316bb127af08cbc` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:41.585964 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=47bc58f24f25465fb316bb127af08cbc)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:41.588053 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:41.638228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '47bc58f24f25465fb316bb127af08cbc', 'request_id': 'req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:41.646592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '47bc58f24f25465fb316bb127af08cbc', 'request_id': 'req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:41.653462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '47bc58f24f25465fb316bb127af08cbc', 'request_id': 'req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:41.664683 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2 admin admin] The token cache is being invalidate because project 47bc58f24f25465fb316bb127af08cbc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:15:41.664970 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:41.665639 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:41.667266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1306/2616] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:15:41 2022] DELETE /identity/v3/projects/47bc58f24f25465fb316bb127af08cbc => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:41.671691 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b9d5b-928b-4352-8fc3-09c73674d5a2 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:41.682338 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7b9d5b-928b-4352-8fc3-09c73674d5a2 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:41.702238 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:41.742116 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2537843a-be7e-4e7c-92e3-67c53fd1ffb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:41.743003 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537843a-be7e-4e7c-92e3-67c53fd1ffb9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:41.743344 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537843a-be7e-4e7c-92e3-67c53fd1ffb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:41.743604 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2537843a-be7e-4e7c-92e3-67c53fd1ffb9 admin admin] PATH_INFO: `/v3/users/ccff097ab42f42b490ae7defcfa43706` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:41.744852 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537843a-be7e-4e7c-92e3-67c53fd1ffb9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ccff097ab42f42b490ae7defcfa43706)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:41.746803 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2537843a-be7e-4e7c-92e3-67c53fd1ffb9 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:41.796403 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2537843a-be7e-4e7c-92e3-67c53fd1ffb9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ccff097ab42f42b490ae7defcfa43706'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:41.797050 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2537843a-be7e-4e7c-92e3-67c53fd1ffb9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ccff097ab42f42b490ae7defcfa43706'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:41.809326 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2537843a-be7e-4e7c-92e3-67c53fd1ffb9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ccff097ab42f42b490ae7defcfa43706'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:41.817580 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2537843a-be7e-4e7c-92e3-67c53fd1ffb9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ccff097ab42f42b490ae7defcfa43706'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:41.818310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-2537843a-be7e-4e7c-92e3-67c53fd1ffb9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ccff097ab42f42b490ae7defcfa43706'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:41.831260 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1311/2617] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:15:41 2022] DELETE /identity/v3/users/ccff097ab42f42b490ae7defcfa43706 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:42.108460 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:42.110049 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:42.124709 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e80f4b9-aa59-49b6-9cef-78942d8da2b2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:42.194350 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8112ca77-c9f6-4cfb-a134-b1965b31bcc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:42.195219 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112ca77-c9f6-4cfb-a134-b1965b31bcc8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:42.195516 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112ca77-c9f6-4cfb-a134-b1965b31bcc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:42.195803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8112ca77-c9f6-4cfb-a134-b1965b31bcc8 admin admin] PATH_INFO: `/v3/projects/6eba0ee08ef345a29dd52abf8d563095` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:42.197270 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112ca77-c9f6-4cfb-a134-b1965b31bcc8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6eba0ee08ef345a29dd52abf8d563095)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:42.199276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8112ca77-c9f6-4cfb-a134-b1965b31bcc8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:42.213926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:42.255398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8112ca77-c9f6-4cfb-a134-b1965b31bcc8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6eba0ee08ef345a29dd52abf8d563095', 'request_id': 'req-8112ca77-c9f6-4cfb-a134-b1965b31bcc8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:42.263616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8112ca77-c9f6-4cfb-a134-b1965b31bcc8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6eba0ee08ef345a29dd52abf8d563095', 'request_id': 'req-8112ca77-c9f6-4cfb-a134-b1965b31bcc8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:42.270192 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8112ca77-c9f6-4cfb-a134-b1965b31bcc8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6eba0ee08ef345a29dd52abf8d563095', 'request_id': 'req-8112ca77-c9f6-4cfb-a134-b1965b31bcc8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:42.281058 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8112ca77-c9f6-4cfb-a134-b1965b31bcc8 admin admin] The token cache is being invalidate because project 6eba0ee08ef345a29dd52abf8d563095 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:15:42.281370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8112ca77-c9f6-4cfb-a134-b1965b31bcc8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:42.282073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-8112ca77-c9f6-4cfb-a134-b1965b31bcc8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:42.284181 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1307/2618] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:15:42 2022] DELETE /identity/v3/projects/6eba0ee08ef345a29dd52abf8d563095 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:42.290183 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36872061-942e-43b3-9a2d-fac5cd8439d1 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:42.290709 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36872061-942e-43b3-9a2d-fac5cd8439d1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:42.290804 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36872061-942e-43b3-9a2d-fac5cd8439d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:42.310729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-36872061-942e-43b3-9a2d-fac5cd8439d1 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:42.311982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-36872061-942e-43b3-9a2d-fac5cd8439d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:42.370231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1312/2619] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:42.378992 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4750d99-99ee-45b7-b530-666b331800c9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:42.379296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4750d99-99ee-45b7-b530-666b331800c9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:42.379581 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4750d99-99ee-45b7-b530-666b331800c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:42.399466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-d4750d99-99ee-45b7-b530-666b331800c9 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:42.400740 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d4750d99-99ee-45b7-b530-666b331800c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:42.409000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1308/2620] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:42.413391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36872061-942e-43b3-9a2d-fac5cd8439d1 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:42.423217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36872061-942e-43b3-9a2d-fac5cd8439d1 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:42.428427 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb211cc-543b-4e23-b4a7-eb564d72db92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:42.429091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb211cc-543b-4e23-b4a7-eb564d72db92 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:42.429422 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb211cc-543b-4e23-b4a7-eb564d72db92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:42.429714 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdb211cc-543b-4e23-b4a7-eb564d72db92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:42.430462 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb211cc-543b-4e23-b4a7-eb564d72db92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:15:42.430784 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb211cc-543b-4e23-b4a7-eb564d72db92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:42.432479 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdb211cc-543b-4e23-b4a7-eb564d72db92 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:42.444295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1313/2621] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:15:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:15:42.448884 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4750d99-99ee-45b7-b530-666b331800c9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:42.458572 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4750d99-99ee-45b7-b530-666b331800c9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:42.464277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d6d05-0e6d-4001-8931-0640ae757d93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:42.464952 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d6d05-0e6d-4001-8931-0640ae757d93 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:42.465243 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d6d05-0e6d-4001-8931-0640ae757d93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:42.465530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f90d6d05-0e6d-4001-8931-0640ae757d93 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:42.466245 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d6d05-0e6d-4001-8931-0640ae757d93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:42.468195 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f90d6d05-0e6d-4001-8931-0640ae757d93 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:42.500341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1309/2622] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:15:42 2022] POST /identity/v3/projects => generated 377 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:42.506647 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb211cc-543b-4e23-b4a7-eb564d72db92 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:42.517445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdb211cc-543b-4e23-b4a7-eb564d72db92 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:42.544251 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:42.579590 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5170e6c-f593-41b4-ad67-a786650304b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:42.580448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5170e6c-f593-41b4-ad67-a786650304b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:42.580739 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5170e6c-f593-41b4-ad67-a786650304b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:42.581017 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5170e6c-f593-41b4-ad67-a786650304b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:42.581743 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5170e6c-f593-41b4-ad67-a786650304b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:42.583753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5170e6c-f593-41b4-ad67-a786650304b7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:42.606202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1314/2623] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:15:42 2022] POST /identity/v3/users => generated 289 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:15:42.610891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d6d05-0e6d-4001-8931-0640ae757d93 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:42.621855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f90d6d05-0e6d-4001-8931-0640ae757d93 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:42.627311 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1036e0-1aea-41bb-8391-8e3c7370cb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:42.628017 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1036e0-1aea-41bb-8391-8e3c7370cb94 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:42.628308 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1036e0-1aea-41bb-8391-8e3c7370cb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:42.628588 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d1036e0-1aea-41bb-8391-8e3c7370cb94 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:42.629316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1036e0-1aea-41bb-8391-8e3c7370cb94 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:42.630929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d1036e0-1aea-41bb-8391-8e3c7370cb94 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:42.644942 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1310/2624] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:42.649808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5170e6c-f593-41b4-ad67-a786650304b7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:42.665194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5170e6c-f593-41b4-ad67-a786650304b7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:42.671856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec237872-2553-4e67-9ef3-4f08d64569c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:42.672518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec237872-2553-4e67-9ef3-4f08d64569c2 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:42.672807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec237872-2553-4e67-9ef3-4f08d64569c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:42.673081 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec237872-2553-4e67-9ef3-4f08d64569c2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:42.673801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec237872-2553-4e67-9ef3-4f08d64569c2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:42.675407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec237872-2553-4e67-9ef3-4f08d64569c2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:42.689764 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1315/2625] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:15:42 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:15:42.696224 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1036e0-1aea-41bb-8391-8e3c7370cb94 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:42.706803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d1036e0-1aea-41bb-8391-8e3c7370cb94 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:42.712004 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbb5ea-a6b9-4302-a0ef-9130315c704c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:42.712927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbb5ea-a6b9-4302-a0ef-9130315c704c admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:42.713221 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbb5ea-a6b9-4302-a0ef-9130315c704c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:42.713504 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fdbb5ea-a6b9-4302-a0ef-9130315c704c admin admin] PATH_INFO: `/v3/projects/1641c9d598d04c00aeff2591c378f784/users/cb701ce5490d46bcb2b36592167289fb/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:42.728811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbb5ea-a6b9-4302-a0ef-9130315c704c admin admin] RBAC: Authorizing `identity:create_grant(project_id=1641c9d598d04c00aeff2591c378f784, user_id=cb701ce5490d46bcb2b36592167289fb, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:42.731475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fdbb5ea-a6b9-4302-a0ef-9130315c704c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:42.747553 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1311/2626] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:15:42 2022] PUT /identity/v3/projects/1641c9d598d04c00aeff2591c378f784/users/cb701ce5490d46bcb2b36592167289fb/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:42.760327 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846d573-0610-45b2-8944-2181b7073889 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:42.761380 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846d573-0610-45b2-8944-2181b7073889 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:42.761807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e846d573-0610-45b2-8944-2181b7073889 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:42.790984 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e846d573-0610-45b2-8944-2181b7073889 None None] MFA Rules not processed for user `cb701ce5490d46bcb2b36592167289fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:42.792245 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e846d573-0610-45b2-8944-2181b7073889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:42.857951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1316/2627] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:42.868771 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbb5ea-a6b9-4302-a0ef-9130315c704c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:42.872006 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7fdbb5ea-a6b9-4302-a0ef-9130315c704c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:42.943004 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fdbb5ea-a6b9-4302-a0ef-9130315c704c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:42.946855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e27e8e-94a4-4d5f-a2df-8f732b30bc48 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '196dc406a95444c39babc50d59f9eb6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:42.947562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e27e8e-94a4-4d5f-a2df-8f732b30bc48 service glance] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:42.947889 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e27e8e-94a4-4d5f-a2df-8f732b30bc48 service glance] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:42.948177 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e27e8e-94a4-4d5f-a2df-8f732b30bc48 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:42.956972 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e27e8e-94a4-4d5f-a2df-8f732b30bc48 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:42.958839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e27e8e-94a4-4d5f-a2df-8f732b30bc48 service glance] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:42.971008 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1312/2628] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:42 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:46.052860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846d573-0610-45b2-8944-2181b7073889 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:46.054279 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e846d573-0610-45b2-8944-2181b7073889 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:46.119161 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e846d573-0610-45b2-8944-2181b7073889 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:46.122687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f9eac-79b7-475a-ade5-d2ff393dea1c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:46.123347 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f9eac-79b7-475a-ade5-d2ff393dea1c service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:46.123634 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f9eac-79b7-475a-ade5-d2ff393dea1c service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:46.123970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-244f9eac-79b7-475a-ade5-d2ff393dea1c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:46.125704 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-244f9eac-79b7-475a-ade5-d2ff393dea1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:46.175947 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:46.193269 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f9eac-79b7-475a-ade5-d2ff393dea1c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:46.194660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-244f9eac-79b7-475a-ade5-d2ff393dea1c service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:46.204999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1317/2629] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:46 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:47.811224 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e27e8e-94a4-4d5f-a2df-8f732b30bc48 service glance] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:47.812876 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-39e27e8e-94a4-4d5f-a2df-8f732b30bc48 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:47.860179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:47.893135 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e27e8e-94a4-4d5f-a2df-8f732b30bc48 service glance] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:47.899618 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438994d-efda-4f7b-9e9e-6088501c07a4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:47.900376 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438994d-efda-4f7b-9e9e-6088501c07a4 service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:47.900668 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438994d-efda-4f7b-9e9e-6088501c07a4 service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:47.900944 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a438994d-efda-4f7b-9e9e-6088501c07a4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:47.903184 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a438994d-efda-4f7b-9e9e-6088501c07a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:47.916243 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438994d-efda-4f7b-9e9e-6088501c07a4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:47.917964 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a438994d-efda-4f7b-9e9e-6088501c07a4 service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:47.929008 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1313/2630] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:47 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:56.200200 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=6 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:57.924345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=16 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:58.379663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f9eac-79b7-475a-ade5-d2ff393dea1c service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:58.381143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-244f9eac-79b7-475a-ade5-d2ff393dea1c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:58.454877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-244f9eac-79b7-475a-ade5-d2ff393dea1c service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:58.460928 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02159fcc-6c8d-4b38-9939-e06cd1ed9a59 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b738f1b8c64590955a640c461b4ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:58.461637 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02159fcc-6c8d-4b38-9939-e06cd1ed9a59 service placement] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:58.461970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02159fcc-6c8d-4b38-9939-e06cd1ed9a59 service placement] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:58.462291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02159fcc-6c8d-4b38-9939-e06cd1ed9a59 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:58.464399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-02159fcc-6c8d-4b38-9939-e06cd1ed9a59 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:58.477032 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02159fcc-6c8d-4b38-9939-e06cd1ed9a59 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:58.479005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02159fcc-6c8d-4b38-9939-e06cd1ed9a59 service placement] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:58.494088 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1318/2631] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:58.802346 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438994d-efda-4f7b-9e9e-6088501c07a4 service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:58.806552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a438994d-efda-4f7b-9e9e-6088501c07a4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:58.874966 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a438994d-efda-4f7b-9e9e-6088501c07a4 service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:58.879946 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b3a8e-1146-42b7-90e8-e90c3f0d0e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:58.880825 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b3a8e-1146-42b7-90e8-e90c3f0d0e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:58.881113 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b3a8e-1146-42b7-90e8-e90c3f0d0e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:58.881390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d8b3a8e-1146-42b7-90e8-e90c3f0d0e0e admin admin] PATH_INFO: `/v3/users/f88bec9e63f44fa089246749e58ca0df` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:58.882645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b3a8e-1146-42b7-90e8-e90c3f0d0e0e admin admin] RBAC: Authorizing `identity:delete_user(user_id=f88bec9e63f44fa089246749e58ca0df)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:58.884661 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d8b3a8e-1146-42b7-90e8-e90c3f0d0e0e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:58.943625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6d8b3a8e-1146-42b7-90e8-e90c3f0d0e0e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f88bec9e63f44fa089246749e58ca0df'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:58.957086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6d8b3a8e-1146-42b7-90e8-e90c3f0d0e0e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f88bec9e63f44fa089246749e58ca0df'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:58.966499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6d8b3a8e-1146-42b7-90e8-e90c3f0d0e0e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f88bec9e63f44fa089246749e58ca0df'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:58.967146 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6d8b3a8e-1146-42b7-90e8-e90c3f0d0e0e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f88bec9e63f44fa089246749e58ca0df'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:58.974581 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-6d8b3a8e-1146-42b7-90e8-e90c3f0d0e0e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f88bec9e63f44fa089246749e58ca0df'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:58.980733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1314/2632] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:15:58 2022] DELETE /identity/v3/users/f88bec9e63f44fa089246749e58ca0df => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:58.985274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf38066-43ed-475e-a801-cd02e3033a94 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:58.985573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf38066-43ed-475e-a801-cd02e3033a94 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:58.985857 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaf38066-43ed-475e-a801-cd02e3033a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:59.004074 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-aaf38066-43ed-475e-a801-cd02e3033a94 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:15:59.005284 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-aaf38066-43ed-475e-a801-cd02e3033a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:59.067904 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1319/2633] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:15:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:15:59.081302 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b3a8e-1146-42b7-90e8-e90c3f0d0e0e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:59.083887 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-6d8b3a8e-1146-42b7-90e8-e90c3f0d0e0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:59.148712 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d8b3a8e-1146-42b7-90e8-e90c3f0d0e0e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:59.154116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9e867-03a0-49eb-9ec2-d998b88a4f6e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:59.154776 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9e867-03a0-49eb-9ec2-d998b88a4f6e service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:59.155067 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9e867-03a0-49eb-9ec2-d998b88a4f6e service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:59.155350 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f9e867-03a0-49eb-9ec2-d998b88a4f6e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:59.165311 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9e867-03a0-49eb-9ec2-d998b88a4f6e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:59.166765 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f9e867-03a0-49eb-9ec2-d998b88a4f6e service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:59.176351 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1315/2634] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:15:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:15:59.462804 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf38066-43ed-475e-a801-cd02e3033a94 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:59.466117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-aaf38066-43ed-475e-a801-cd02e3033a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:15:59.481293 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaf38066-43ed-475e-a801-cd02e3033a94 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:59.486318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030e8c5-acea-4453-acc6-c9459a6b9ee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:59.487250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030e8c5-acea-4453-acc6-c9459a6b9ee6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:59.487636 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030e8c5-acea-4453-acc6-c9459a6b9ee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:59.488094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8030e8c5-acea-4453-acc6-c9459a6b9ee6 admin admin] PATH_INFO: `/v3/projects/e7bac61121f6455e8b9cccd259a8da85` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:59.489428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030e8c5-acea-4453-acc6-c9459a6b9ee6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e7bac61121f6455e8b9cccd259a8da85)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:59.491952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8030e8c5-acea-4453-acc6-c9459a6b9ee6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:59.543407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8030e8c5-acea-4453-acc6-c9459a6b9ee6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e7bac61121f6455e8b9cccd259a8da85', 'request_id': 'req-8030e8c5-acea-4453-acc6-c9459a6b9ee6'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:59.549985 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8030e8c5-acea-4453-acc6-c9459a6b9ee6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e7bac61121f6455e8b9cccd259a8da85', 'request_id': 'req-8030e8c5-acea-4453-acc6-c9459a6b9ee6'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:59.559081 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8030e8c5-acea-4453-acc6-c9459a6b9ee6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e7bac61121f6455e8b9cccd259a8da85', 'request_id': 'req-8030e8c5-acea-4453-acc6-c9459a6b9ee6'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:59.570622 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8030e8c5-acea-4453-acc6-c9459a6b9ee6 admin admin] The token cache is being invalidate because project e7bac61121f6455e8b9cccd259a8da85 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:15:59.570913 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8030e8c5-acea-4453-acc6-c9459a6b9ee6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:59.571603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8030e8c5-acea-4453-acc6-c9459a6b9ee6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:59.573219 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1320/2635] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:15:59 2022] DELETE /identity/v3/projects/e7bac61121f6455e8b9cccd259a8da85 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:15:59.577379 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9e867-03a0-49eb-9ec2-d998b88a4f6e service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:15:59.588608 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f9e867-03a0-49eb-9ec2-d998b88a4f6e service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:15:59.619562 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:15:59.656700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-918b27b6-92ab-467b-83bb-7bc59aec8388 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:15:59.657598 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b27b6-92ab-467b-83bb-7bc59aec8388 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:15:59.657894 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b27b6-92ab-467b-83bb-7bc59aec8388 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:15:59.658178 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-918b27b6-92ab-467b-83bb-7bc59aec8388 admin admin] PATH_INFO: `/v3/users/6c731507d7054573a82cc62e9214dfa6` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:15:59.659373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b27b6-92ab-467b-83bb-7bc59aec8388 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6c731507d7054573a82cc62e9214dfa6)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:15:59.661366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-918b27b6-92ab-467b-83bb-7bc59aec8388 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:15:59.711603 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-918b27b6-92ab-467b-83bb-7bc59aec8388 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6c731507d7054573a82cc62e9214dfa6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:59.723548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-918b27b6-92ab-467b-83bb-7bc59aec8388 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6c731507d7054573a82cc62e9214dfa6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:59.733284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-918b27b6-92ab-467b-83bb-7bc59aec8388 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6c731507d7054573a82cc62e9214dfa6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:59.734085 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-918b27b6-92ab-467b-83bb-7bc59aec8388 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6c731507d7054573a82cc62e9214dfa6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:59.741050 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-918b27b6-92ab-467b-83bb-7bc59aec8388 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6c731507d7054573a82cc62e9214dfa6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:15:59.747068 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1316/2636] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:15:59 2022] DELETE /identity/v3/users/6c731507d7054573a82cc62e9214dfa6 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:00.026999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030e8c5-acea-4453-acc6-c9459a6b9ee6 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:00.028582 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8030e8c5-acea-4453-acc6-c9459a6b9ee6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:00.048130 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8030e8c5-acea-4453-acc6-c9459a6b9ee6 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:00.058013 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:00.115685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6048a3aa-5ecb-46b0-85fa-625cc83d8adb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:00.116551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048a3aa-5ecb-46b0-85fa-625cc83d8adb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:00.116841 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048a3aa-5ecb-46b0-85fa-625cc83d8adb admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:00.117127 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6048a3aa-5ecb-46b0-85fa-625cc83d8adb admin admin] PATH_INFO: `/v3/projects/2a9084e3584f44cb844b6af374f6eb71` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:00.118422 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048a3aa-5ecb-46b0-85fa-625cc83d8adb admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a9084e3584f44cb844b6af374f6eb71)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:00.120450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6048a3aa-5ecb-46b0-85fa-625cc83d8adb admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:00.170675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6048a3aa-5ecb-46b0-85fa-625cc83d8adb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a9084e3584f44cb844b6af374f6eb71', 'request_id': 'req-6048a3aa-5ecb-46b0-85fa-625cc83d8adb'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:00.179367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6048a3aa-5ecb-46b0-85fa-625cc83d8adb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a9084e3584f44cb844b6af374f6eb71', 'request_id': 'req-6048a3aa-5ecb-46b0-85fa-625cc83d8adb'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:00.191377 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6048a3aa-5ecb-46b0-85fa-625cc83d8adb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a9084e3584f44cb844b6af374f6eb71', 'request_id': 'req-6048a3aa-5ecb-46b0-85fa-625cc83d8adb'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:00.206295 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6048a3aa-5ecb-46b0-85fa-625cc83d8adb admin admin] The token cache is being invalidate because project 2a9084e3584f44cb844b6af374f6eb71 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:00.206592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6048a3aa-5ecb-46b0-85fa-625cc83d8adb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:00.207413 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6048a3aa-5ecb-46b0-85fa-625cc83d8adb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:00.209494 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1321/2637] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:00 2022] DELETE /identity/v3/projects/2a9084e3584f44cb844b6af374f6eb71 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:00.214841 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5a229-e62b-4b10-a846-c7cc9b2c86e1 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:00.215334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5a229-e62b-4b10-a846-c7cc9b2c86e1 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:00.215463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a5a229-e62b-4b10-a846-c7cc9b2c86e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:00.234457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-73a5a229-e62b-4b10-a846-c7cc9b2c86e1 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:00.235666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-73a5a229-e62b-4b10-a846-c7cc9b2c86e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:00.302594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1317/2638] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:00.305777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0851ac-9075-4346-a316-29e9dbfea5d6 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:00.306101 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0851ac-9075-4346-a316-29e9dbfea5d6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:00.306351 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0851ac-9075-4346-a316-29e9dbfea5d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:00.327237 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-2f0851ac-9075-4346-a316-29e9dbfea5d6 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:00.328497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-2f0851ac-9075-4346-a316-29e9dbfea5d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:00.337929 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1322/2639] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:00.342550 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5a229-e62b-4b10-a846-c7cc9b2c86e1 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:00.352756 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a5a229-e62b-4b10-a846-c7cc9b2c86e1 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:00.357817 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87260bab-bb5c-449d-9760-3ec8667de18f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:00.358477 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87260bab-bb5c-449d-9760-3ec8667de18f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:00.358765 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87260bab-bb5c-449d-9760-3ec8667de18f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:00.359076 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87260bab-bb5c-449d-9760-3ec8667de18f admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:00.359808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87260bab-bb5c-449d-9760-3ec8667de18f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:16:00.360167 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87260bab-bb5c-449d-9760-3ec8667de18f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:00.361989 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87260bab-bb5c-449d-9760-3ec8667de18f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:00.374156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1318/2640] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:16:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:16:00.379166 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0851ac-9075-4346-a316-29e9dbfea5d6 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:00.392800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0851ac-9075-4346-a316-29e9dbfea5d6 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:00.399772 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7ef94-a22e-4114-a95f-692740d220ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:00.400696 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7ef94-a22e-4114-a95f-692740d220ef admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:00.401053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7ef94-a22e-4114-a95f-692740d220ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:00.401420 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65e7ef94-a22e-4114-a95f-692740d220ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:00.402117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7ef94-a22e-4114-a95f-692740d220ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:00.404042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65e7ef94-a22e-4114-a95f-692740d220ef admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:00.431963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1323/2641] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:00 2022] POST /identity/v3/projects => generated 373 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:16:00.434822 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87260bab-bb5c-449d-9760-3ec8667de18f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:00.445472 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87260bab-bb5c-449d-9760-3ec8667de18f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:00.471931 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:00.505774 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1057e-339c-42e1-8048-ee7e82cd138b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:00.506480 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1057e-339c-42e1-8048-ee7e82cd138b admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:00.506774 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1057e-339c-42e1-8048-ee7e82cd138b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:00.507063 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f1057e-339c-42e1-8048-ee7e82cd138b admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:00.507803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1057e-339c-42e1-8048-ee7e82cd138b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:00.509782 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f1057e-339c-42e1-8048-ee7e82cd138b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:00.532784 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1319/2642] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:00 2022] POST /identity/v3/users => generated 287 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:16:00.537597 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7ef94-a22e-4114-a95f-692740d220ef admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:00.548116 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65e7ef94-a22e-4114-a95f-692740d220ef admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:00.553371 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268f713-f05c-4734-8afd-918dad218fcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:00.554044 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268f713-f05c-4734-8afd-918dad218fcb admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:00.554332 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268f713-f05c-4734-8afd-918dad218fcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:00.554612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1268f713-f05c-4734-8afd-918dad218fcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:00.555346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268f713-f05c-4734-8afd-918dad218fcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:00.556996 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1268f713-f05c-4734-8afd-918dad218fcb admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:00.567841 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1324/2643] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:00.571929 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1057e-339c-42e1-8048-ee7e82cd138b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:00.582606 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f1057e-339c-42e1-8048-ee7e82cd138b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:00.587803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13263768-ff1d-4e8a-a204-2f4057a93ebe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:00.588552 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13263768-ff1d-4e8a-a204-2f4057a93ebe admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:00.588955 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13263768-ff1d-4e8a-a204-2f4057a93ebe admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:00.589239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13263768-ff1d-4e8a-a204-2f4057a93ebe admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:00.589960 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13263768-ff1d-4e8a-a204-2f4057a93ebe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:00.591554 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13263768-ff1d-4e8a-a204-2f4057a93ebe admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:00.602033 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1320/2644] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:00 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:00.606223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268f713-f05c-4734-8afd-918dad218fcb admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:00.617179 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1268f713-f05c-4734-8afd-918dad218fcb admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:00.623481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d93d51-d012-4daf-99c3-31cbee980702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:00.624476 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d93d51-d012-4daf-99c3-31cbee980702 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:00.624669 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d93d51-d012-4daf-99c3-31cbee980702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:00.624986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11d93d51-d012-4daf-99c3-31cbee980702 admin admin] PATH_INFO: `/v3/projects/479d82cafd5642f0ab928c8efccffdd7/users/2041527f7b014f1aa718f08e12c70d75/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:00.641656 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d93d51-d012-4daf-99c3-31cbee980702 admin admin] RBAC: Authorizing `identity:create_grant(project_id=479d82cafd5642f0ab928c8efccffdd7, user_id=2041527f7b014f1aa718f08e12c70d75, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:00.645312 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11d93d51-d012-4daf-99c3-31cbee980702 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:00.659557 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1325/2645] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:00 2022] PUT /identity/v3/projects/479d82cafd5642f0ab928c8efccffdd7/users/2041527f7b014f1aa718f08e12c70d75/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:00.670356 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50d0d1-ef97-4b23-82c7-d7e3d320572a None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:00.670658 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50d0d1-ef97-4b23-82c7-d7e3d320572a None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:00.670951 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff50d0d1-ef97-4b23-82c7-d7e3d320572a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:00.700710 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ff50d0d1-ef97-4b23-82c7-d7e3d320572a None None] MFA Rules not processed for user `2041527f7b014f1aa718f08e12c70d75`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:00.702001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ff50d0d1-ef97-4b23-82c7-d7e3d320572a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:00.784820 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1321/2646] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:00 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:00.791860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d93d51-d012-4daf-99c3-31cbee980702 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:00.844288 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:00.866375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11d93d51-d012-4daf-99c3-31cbee980702 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:00.871238 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82a771-3804-40f7-bcdf-0fe20b48aab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:00.871914 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82a771-3804-40f7-bcdf-0fe20b48aab6 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:00.872201 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82a771-3804-40f7-bcdf-0fe20b48aab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:00.872479 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff82a771-3804-40f7-bcdf-0fe20b48aab6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:00.873228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82a771-3804-40f7-bcdf-0fe20b48aab6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:00.875231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff82a771-3804-40f7-bcdf-0fe20b48aab6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:00.908843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1326/2647] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:00 2022] POST /identity/v3/projects => generated 371 bytes in 118 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:00.914799 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50d0d1-ef97-4b23-82c7-d7e3d320572a None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:00.927236 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff50d0d1-ef97-4b23-82c7-d7e3d320572a None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:00.993274 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2c8e0-a110-45ce-9c1c-ee097fdfba72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:00.993959 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2c8e0-a110-45ce-9c1c-ee097fdfba72 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:00.994261 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2c8e0-a110-45ce-9c1c-ee097fdfba72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:00.994543 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b2c8e0-a110-45ce-9c1c-ee097fdfba72 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:00.995261 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2c8e0-a110-45ce-9c1c-ee097fdfba72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:00.997277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b2c8e0-a110-45ce-9c1c-ee097fdfba72 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:01.020427 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1322/2648] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:00 2022] POST /identity/v3/users => generated 292 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:01.024766 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82a771-3804-40f7-bcdf-0fe20b48aab6 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:01.035878 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff82a771-3804-40f7-bcdf-0fe20b48aab6 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:01.041046 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180739c7-9477-49c0-bff6-f9014bf22b79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:01.041724 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180739c7-9477-49c0-bff6-f9014bf22b79 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:01.042007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180739c7-9477-49c0-bff6-f9014bf22b79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:01.042361 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180739c7-9477-49c0-bff6-f9014bf22b79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:01.043093 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180739c7-9477-49c0-bff6-f9014bf22b79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:01.044807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180739c7-9477-49c0-bff6-f9014bf22b79 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:01.055908 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1327/2649] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:01.060329 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2c8e0-a110-45ce-9c1c-ee097fdfba72 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:01.071137 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b2c8e0-a110-45ce-9c1c-ee097fdfba72 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:01.076646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9fa203-17ba-4adf-b88a-a57b9c733a0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:01.077548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9fa203-17ba-4adf-b88a-a57b9c733a0c admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:01.077836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9fa203-17ba-4adf-b88a-a57b9c733a0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:01.078112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9fa203-17ba-4adf-b88a-a57b9c733a0c admin admin] PATH_INFO: `/v3/projects/f264dfc111204ddcb90ede20d93ed1e6/users/f2a66350efab4b96a454d83f415eb784/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:01.093731 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9fa203-17ba-4adf-b88a-a57b9c733a0c admin admin] RBAC: Authorizing `identity:create_grant(project_id=f264dfc111204ddcb90ede20d93ed1e6, user_id=f2a66350efab4b96a454d83f415eb784, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:01.096429 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9fa203-17ba-4adf-b88a-a57b9c733a0c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:01.112547 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1323/2650] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:01 2022] PUT /identity/v3/projects/f264dfc111204ddcb90ede20d93ed1e6/users/f2a66350efab4b96a454d83f415eb784/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:01.117122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180739c7-9477-49c0-bff6-f9014bf22b79 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:01.128031 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180739c7-9477-49c0-bff6-f9014bf22b79 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:01.193909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c83e0ec-a229-4bd9-8d72-8ccd5122c0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:01.194600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83e0ec-a229-4bd9-8d72-8ccd5122c0c8 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:01.194898 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83e0ec-a229-4bd9-8d72-8ccd5122c0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:01.195183 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c83e0ec-a229-4bd9-8d72-8ccd5122c0c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:01.195967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83e0ec-a229-4bd9-8d72-8ccd5122c0c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:01.197603 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c83e0ec-a229-4bd9-8d72-8ccd5122c0c8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:01.207733 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1328/2651] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:01 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:01.212177 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9fa203-17ba-4adf-b88a-a57b9c733a0c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:01.217946 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:01.223915 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9fa203-17ba-4adf-b88a-a57b9c733a0c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:01.229847 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d667b1-3746-4f6b-8af4-1c84859be5d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:01.230750 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d667b1-3746-4f6b-8af4-1c84859be5d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:01.231041 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d667b1-3746-4f6b-8af4-1c84859be5d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:01.231325 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88d667b1-3746-4f6b-8af4-1c84859be5d1 admin admin] PATH_INFO: `/v3/projects/f264dfc111204ddcb90ede20d93ed1e6/users/f2a66350efab4b96a454d83f415eb784/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:01.233402 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d667b1-3746-4f6b-8af4-1c84859be5d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f264dfc111204ddcb90ede20d93ed1e6, user_id=f2a66350efab4b96a454d83f415eb784, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:01.236135 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88d667b1-3746-4f6b-8af4-1c84859be5d1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:01.246674 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1324/2652] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:01 2022] PUT /identity/v3/projects/f264dfc111204ddcb90ede20d93ed1e6/users/f2a66350efab4b96a454d83f415eb784/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:01.258196 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b717213-f722-43f8-a588-815c1533d694 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:01.258488 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b717213-f722-43f8-a588-815c1533d694 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:01.258770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b717213-f722-43f8-a588-815c1533d694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:01.285773 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8b717213-f722-43f8-a588-815c1533d694 None None] MFA Rules not processed for user `f2a66350efab4b96a454d83f415eb784`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:01.287109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8b717213-f722-43f8-a588-815c1533d694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:01.367772 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1329/2653] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:01 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:01.380377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d667b1-3746-4f6b-8af4-1c84859be5d1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:01.381840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-88d667b1-3746-4f6b-8af4-1c84859be5d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:01.442288 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88d667b1-3746-4f6b-8af4-1c84859be5d1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:01.446692 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c88be213-3978-457f-b6d5-38366b136843 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:01.447336 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88be213-3978-457f-b6d5-38366b136843 service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:01.447624 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88be213-3978-457f-b6d5-38366b136843 service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:01.447937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c88be213-3978-457f-b6d5-38366b136843 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:01.457802 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88be213-3978-457f-b6d5-38366b136843 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:01.459239 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c88be213-3978-457f-b6d5-38366b136843 service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:01.468910 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1325/2654] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:01 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:01.848194 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b717213-f722-43f8-a588-815c1533d694 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:01.889126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:01.913957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b717213-f722-43f8-a588-815c1533d694 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:01.918568 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6b605-48c5-41a0-b49a-25992888d341 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:01.919456 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6b605-48c5-41a0-b49a-25992888d341 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:01.919741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6b605-48c5-41a0-b49a-25992888d341 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:01.920176 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb6b605-48c5-41a0-b49a-25992888d341 admin admin] PATH_INFO: `/v3/users/2041527f7b014f1aa718f08e12c70d75` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:01.921361 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb6b605-48c5-41a0-b49a-25992888d341 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2041527f7b014f1aa718f08e12c70d75)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:01.923303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb6b605-48c5-41a0-b49a-25992888d341 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:01.970827 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9eb6b605-48c5-41a0-b49a-25992888d341 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2041527f7b014f1aa718f08e12c70d75'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:01.971491 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9eb6b605-48c5-41a0-b49a-25992888d341 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2041527f7b014f1aa718f08e12c70d75'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:01.983073 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9eb6b605-48c5-41a0-b49a-25992888d341 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2041527f7b014f1aa718f08e12c70d75'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:01.991484 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9eb6b605-48c5-41a0-b49a-25992888d341 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2041527f7b014f1aa718f08e12c70d75'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:01.992218 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9eb6b605-48c5-41a0-b49a-25992888d341 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2041527f7b014f1aa718f08e12c70d75'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:02.005851 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1330/2655] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:16:01 2022] DELETE /identity/v3/users/2041527f7b014f1aa718f08e12c70d75 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:02.010549 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c11939-5907-4b49-b411-d20ec4d1b127 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:02.010908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c11939-5907-4b49-b411-d20ec4d1b127 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:02.011212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c11939-5907-4b49-b411-d20ec4d1b127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:02.029242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f2c11939-5907-4b49-b411-d20ec4d1b127 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:02.030467 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f2c11939-5907-4b49-b411-d20ec4d1b127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:02.094933 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1326/2656] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:02.105049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6b605-48c5-41a0-b49a-25992888d341 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:02.109593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9eb6b605-48c5-41a0-b49a-25992888d341 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:02.181313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb6b605-48c5-41a0-b49a-25992888d341 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:02.186071 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91275c09-654e-4ceb-a8de-43cd6e34595b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:02.186844 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91275c09-654e-4ceb-a8de-43cd6e34595b service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:02.187132 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91275c09-654e-4ceb-a8de-43cd6e34595b service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:02.187429 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91275c09-654e-4ceb-a8de-43cd6e34595b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:02.197190 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91275c09-654e-4ceb-a8de-43cd6e34595b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:02.198703 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91275c09-654e-4ceb-a8de-43cd6e34595b service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:02.208607 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1331/2657] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:02.498332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c11939-5907-4b49-b411-d20ec4d1b127 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:02.500137 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f2c11939-5907-4b49-b411-d20ec4d1b127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:02.514571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c11939-5907-4b49-b411-d20ec4d1b127 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:02.519447 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:02.520366 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:02.520673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:02.520945 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83 admin admin] PATH_INFO: `/v3/projects/479d82cafd5642f0ab928c8efccffdd7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:02.522178 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83 admin admin] RBAC: Authorizing `identity:delete_project(project_id=479d82cafd5642f0ab928c8efccffdd7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:02.524131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:02.571974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '479d82cafd5642f0ab928c8efccffdd7', 'request_id': 'req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:02.583583 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '479d82cafd5642f0ab928c8efccffdd7', 'request_id': 'req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:02.590407 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '479d82cafd5642f0ab928c8efccffdd7', 'request_id': 'req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:02.601373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83 admin admin] The token cache is being invalidate because project 479d82cafd5642f0ab928c8efccffdd7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:02.601675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:02.602392 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:02.604101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1327/2658] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:02 2022] DELETE /identity/v3/projects/479d82cafd5642f0ab928c8efccffdd7 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:02.608593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91275c09-654e-4ceb-a8de-43cd6e34595b service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:02.618640 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91275c09-654e-4ceb-a8de-43cd6e34595b service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:02.672739 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:02.681270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed062349-d9a4-4fa3-9760-8dad3eb1020b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:02.682188 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed062349-d9a4-4fa3-9760-8dad3eb1020b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:02.682476 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed062349-d9a4-4fa3-9760-8dad3eb1020b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:02.682802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed062349-d9a4-4fa3-9760-8dad3eb1020b admin admin] PATH_INFO: `/v3/users/f2a66350efab4b96a454d83f415eb784` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:02.684228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed062349-d9a4-4fa3-9760-8dad3eb1020b admin admin] RBAC: Authorizing `identity:delete_user(user_id=f2a66350efab4b96a454d83f415eb784)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:02.687114 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed062349-d9a4-4fa3-9760-8dad3eb1020b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:02.739944 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ed062349-d9a4-4fa3-9760-8dad3eb1020b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f2a66350efab4b96a454d83f415eb784'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:02.740655 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ed062349-d9a4-4fa3-9760-8dad3eb1020b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f2a66350efab4b96a454d83f415eb784'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:02.752683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ed062349-d9a4-4fa3-9760-8dad3eb1020b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f2a66350efab4b96a454d83f415eb784'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:02.761654 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ed062349-d9a4-4fa3-9760-8dad3eb1020b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f2a66350efab4b96a454d83f415eb784'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:02.762274 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ed062349-d9a4-4fa3-9760-8dad3eb1020b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f2a66350efab4b96a454d83f415eb784'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:02.774598 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1332/2659] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:16:02 2022] DELETE /identity/v3/users/f2a66350efab4b96a454d83f415eb784 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:03.058717 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:03.060536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:03.074276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbe9d58-c247-45d3-9ad9-19db0cfaca83 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:03.087663 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:03.137722 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82569d0d-e06d-4b42-be1b-0fcd13fde8da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:03.138598 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82569d0d-e06d-4b42-be1b-0fcd13fde8da admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:03.138889 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82569d0d-e06d-4b42-be1b-0fcd13fde8da admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:03.139171 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82569d0d-e06d-4b42-be1b-0fcd13fde8da admin admin] PATH_INFO: `/v3/projects/f264dfc111204ddcb90ede20d93ed1e6` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:03.140554 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82569d0d-e06d-4b42-be1b-0fcd13fde8da admin admin] RBAC: Authorizing `identity:delete_project(project_id=f264dfc111204ddcb90ede20d93ed1e6)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:03.142480 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82569d0d-e06d-4b42-be1b-0fcd13fde8da admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:03.192842 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-82569d0d-e06d-4b42-be1b-0fcd13fde8da admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f264dfc111204ddcb90ede20d93ed1e6', 'request_id': 'req-82569d0d-e06d-4b42-be1b-0fcd13fde8da'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:03.201834 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-82569d0d-e06d-4b42-be1b-0fcd13fde8da admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f264dfc111204ddcb90ede20d93ed1e6', 'request_id': 'req-82569d0d-e06d-4b42-be1b-0fcd13fde8da'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:03.208367 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-82569d0d-e06d-4b42-be1b-0fcd13fde8da admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f264dfc111204ddcb90ede20d93ed1e6', 'request_id': 'req-82569d0d-e06d-4b42-be1b-0fcd13fde8da'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:03.219233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-82569d0d-e06d-4b42-be1b-0fcd13fde8da admin admin] The token cache is being invalidate because project f264dfc111204ddcb90ede20d93ed1e6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:03.219708 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-82569d0d-e06d-4b42-be1b-0fcd13fde8da admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:03.220483 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-82569d0d-e06d-4b42-be1b-0fcd13fde8da admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:03.222167 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1328/2660] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:03 2022] DELETE /identity/v3/projects/f264dfc111204ddcb90ede20d93ed1e6 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:03.228999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1858dc-86b2-4d08-bb1c-11cbe4ae643a None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:03.229584 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1858dc-86b2-4d08-bb1c-11cbe4ae643a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:03.229769 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1858dc-86b2-4d08-bb1c-11cbe4ae643a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:03.248712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-5a1858dc-86b2-4d08-bb1c-11cbe4ae643a None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:03.249926 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-5a1858dc-86b2-4d08-bb1c-11cbe4ae643a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:03.314660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1333/2661] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:03.322753 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefe839-b58b-421f-aceb-7e283f3928c3 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:03.323056 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefe839-b58b-421f-aceb-7e283f3928c3 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:03.323344 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aefe839-b58b-421f-aceb-7e283f3928c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:03.342832 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3aefe839-b58b-421f-aceb-7e283f3928c3 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:03.344086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3aefe839-b58b-421f-aceb-7e283f3928c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:03.353385 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1329/2662] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:03.357914 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1858dc-86b2-4d08-bb1c-11cbe4ae643a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:03.368141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a1858dc-86b2-4d08-bb1c-11cbe4ae643a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:03.373306 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee902b-1f7a-4de7-bd09-95214c570011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:03.373978 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee902b-1f7a-4de7-bd09-95214c570011 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:03.374266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee902b-1f7a-4de7-bd09-95214c570011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:03.374532 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ee902b-1f7a-4de7-bd09-95214c570011 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:03.375268 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee902b-1f7a-4de7-bd09-95214c570011 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:16:03.375580 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee902b-1f7a-4de7-bd09-95214c570011 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:03.377543 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ee902b-1f7a-4de7-bd09-95214c570011 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:03.389995 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1334/2663] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:16:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:16:03.394503 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aefe839-b58b-421f-aceb-7e283f3928c3 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:03.404521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aefe839-b58b-421f-aceb-7e283f3928c3 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:03.409765 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925df55c-acff-4d59-a4be-ada10fefab4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:03.410412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925df55c-acff-4d59-a4be-ada10fefab4b admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:03.410700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925df55c-acff-4d59-a4be-ada10fefab4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:03.410977 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-925df55c-acff-4d59-a4be-ada10fefab4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:03.411675 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925df55c-acff-4d59-a4be-ada10fefab4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:03.413628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-925df55c-acff-4d59-a4be-ada10fefab4b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:03.441488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1330/2664] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:03 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:03.445713 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee902b-1f7a-4de7-bd09-95214c570011 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:03.455698 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ee902b-1f7a-4de7-bd09-95214c570011 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:03.517269 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533ad93-190a-4b85-a54a-b6edbd67b77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:03.518293 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533ad93-190a-4b85-a54a-b6edbd67b77f admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:03.518712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533ad93-190a-4b85-a54a-b6edbd67b77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:03.519131 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5533ad93-190a-4b85-a54a-b6edbd67b77f admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:03.520228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533ad93-190a-4b85-a54a-b6edbd67b77f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:03.523292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5533ad93-190a-4b85-a54a-b6edbd67b77f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:03.533866 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=87,keystone:DELETE=10,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:03.549362 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1335/2665] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:03 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:03.553432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925df55c-acff-4d59-a4be-ada10fefab4b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:03.564589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-925df55c-acff-4d59-a4be-ada10fefab4b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:03.570681 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b818b6-934f-4524-8969-4a5894f3a8de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:03.571354 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b818b6-934f-4524-8969-4a5894f3a8de admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:03.571651 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b818b6-934f-4524-8969-4a5894f3a8de admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:03.571965 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b818b6-934f-4524-8969-4a5894f3a8de admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:03.572704 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b818b6-934f-4524-8969-4a5894f3a8de admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:03.574325 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b818b6-934f-4524-8969-4a5894f3a8de admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:03.586684 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1331/2666] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:03.591697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533ad93-190a-4b85-a54a-b6edbd67b77f admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:03.601957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5533ad93-190a-4b85-a54a-b6edbd67b77f admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:03.607079 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da89ffc-33aa-4251-8423-c51772e56f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:03.607739 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da89ffc-33aa-4251-8423-c51772e56f85 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:03.608056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da89ffc-33aa-4251-8423-c51772e56f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:03.608345 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da89ffc-33aa-4251-8423-c51772e56f85 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:03.609071 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da89ffc-33aa-4251-8423-c51772e56f85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:03.610692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da89ffc-33aa-4251-8423-c51772e56f85 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:03.621045 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1336/2667] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:03 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:03.625107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b818b6-934f-4524-8969-4a5894f3a8de admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:03.636064 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b818b6-934f-4524-8969-4a5894f3a8de admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:03.641619 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790eaa7-67ae-4ca6-8c58-1d4f3c14c321 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:03.642536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790eaa7-67ae-4ca6-8c58-1d4f3c14c321 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:03.642833 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790eaa7-67ae-4ca6-8c58-1d4f3c14c321 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:03.643117 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5790eaa7-67ae-4ca6-8c58-1d4f3c14c321 admin admin] PATH_INFO: `/v3/projects/72e1d885fefc45b39a6fff0b0c43956f/users/62c666e5e33e4db38cb82a681f9c7fac/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:03.658312 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790eaa7-67ae-4ca6-8c58-1d4f3c14c321 admin admin] RBAC: Authorizing `identity:create_grant(project_id=72e1d885fefc45b39a6fff0b0c43956f, user_id=62c666e5e33e4db38cb82a681f9c7fac, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:03.661066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5790eaa7-67ae-4ca6-8c58-1d4f3c14c321 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:03.676381 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1332/2668] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:03 2022] PUT /identity/v3/projects/72e1d885fefc45b39a6fff0b0c43956f/users/62c666e5e33e4db38cb82a681f9c7fac/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:03.687671 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a879df-87fd-4081-9009-8dd79f71f04c None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:03.688005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a879df-87fd-4081-9009-8dd79f71f04c None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:03.688334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95a879df-87fd-4081-9009-8dd79f71f04c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:03.715693 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-95a879df-87fd-4081-9009-8dd79f71f04c None None] MFA Rules not processed for user `62c666e5e33e4db38cb82a681f9c7fac`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:03.717276 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-95a879df-87fd-4081-9009-8dd79f71f04c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:03.785644 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1337/2669] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:03 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:03.790586 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790eaa7-67ae-4ca6-8c58-1d4f3c14c321 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:03.840112 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:03.860176 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5790eaa7-67ae-4ca6-8c58-1d4f3c14c321 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:03.865187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91fd65-1277-4be0-8260-ee63ca91b2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:03.866147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91fd65-1277-4be0-8260-ee63ca91b2b3 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:03.866539 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91fd65-1277-4be0-8260-ee63ca91b2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:03.866827 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff91fd65-1277-4be0-8260-ee63ca91b2b3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:03.867647 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91fd65-1277-4be0-8260-ee63ca91b2b3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:03.869736 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff91fd65-1277-4be0-8260-ee63ca91b2b3 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:03.899120 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1333/2670] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:03 2022] POST /identity/v3/projects => generated 387 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:03.903301 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a879df-87fd-4081-9009-8dd79f71f04c None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:03.914155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95a879df-87fd-4081-9009-8dd79f71f04c None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:03.972421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f9561-c10a-401e-b2a7-dddad638dd43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:03.973046 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f9561-c10a-401e-b2a7-dddad638dd43 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:03.973337 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f9561-c10a-401e-b2a7-dddad638dd43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:03.973623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-628f9561-c10a-401e-b2a7-dddad638dd43 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:03.974337 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f9561-c10a-401e-b2a7-dddad638dd43 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:03.976277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-628f9561-c10a-401e-b2a7-dddad638dd43 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:03.997155 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1338/2671] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:03 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:04.001849 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91fd65-1277-4be0-8260-ee63ca91b2b3 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:04.012477 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff91fd65-1277-4be0-8260-ee63ca91b2b3 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:04.017258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b600f-cde9-4ec7-9e92-629df61c4287 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:04.017921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b600f-cde9-4ec7-9e92-629df61c4287 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:04.018208 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b600f-cde9-4ec7-9e92-629df61c4287 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:04.018486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b600f-cde9-4ec7-9e92-629df61c4287 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:04.019216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b600f-cde9-4ec7-9e92-629df61c4287 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:04.020947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b600f-cde9-4ec7-9e92-629df61c4287 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:04.033078 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1334/2672] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:16:04.037516 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f9561-c10a-401e-b2a7-dddad638dd43 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:04.048089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-628f9561-c10a-401e-b2a7-dddad638dd43 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:04.053859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74217b-acc2-480b-9a02-c4574a0927b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:04.054761 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74217b-acc2-480b-9a02-c4574a0927b1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:04.055047 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74217b-acc2-480b-9a02-c4574a0927b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:04.055327 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b74217b-acc2-480b-9a02-c4574a0927b1 admin admin] PATH_INFO: `/v3/projects/209eb7e9dcd94e0aa7c0401e18cfe89c/users/7ac1fcb8d0d142739af91ec8aafdeb23/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:04.070562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74217b-acc2-480b-9a02-c4574a0927b1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=209eb7e9dcd94e0aa7c0401e18cfe89c, user_id=7ac1fcb8d0d142739af91ec8aafdeb23, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:04.073306 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b74217b-acc2-480b-9a02-c4574a0927b1 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:04.088217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1339/2673] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:04 2022] PUT /identity/v3/projects/209eb7e9dcd94e0aa7c0401e18cfe89c/users/7ac1fcb8d0d142739af91ec8aafdeb23/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:16:04.092223 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b600f-cde9-4ec7-9e92-629df61c4287 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:04.102956 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b600f-cde9-4ec7-9e92-629df61c4287 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:04.164753 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb702cd8-facc-4ba6-aeb7-bb43d2658d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:04.165678 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb702cd8-facc-4ba6-aeb7-bb43d2658d31 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:04.166080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb702cd8-facc-4ba6-aeb7-bb43d2658d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:04.166437 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb702cd8-facc-4ba6-aeb7-bb43d2658d31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:04.167454 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb702cd8-facc-4ba6-aeb7-bb43d2658d31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:04.169438 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb702cd8-facc-4ba6-aeb7-bb43d2658d31 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:04.183786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1335/2674] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:04 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:04.185823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74217b-acc2-480b-9a02-c4574a0927b1 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:04.196357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b74217b-acc2-480b-9a02-c4574a0927b1 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:04.201019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4a605-5a23-46c2-b735-e7a1ef913743 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:04.201919 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4a605-5a23-46c2-b735-e7a1ef913743 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:04.202207 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4a605-5a23-46c2-b735-e7a1ef913743 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:04.202490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a4a605-5a23-46c2-b735-e7a1ef913743 admin admin] PATH_INFO: `/v3/projects/209eb7e9dcd94e0aa7c0401e18cfe89c/users/7ac1fcb8d0d142739af91ec8aafdeb23/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:04.204675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4a605-5a23-46c2-b735-e7a1ef913743 admin admin] RBAC: Authorizing `identity:create_grant(project_id=209eb7e9dcd94e0aa7c0401e18cfe89c, user_id=7ac1fcb8d0d142739af91ec8aafdeb23, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:04.207336 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a4a605-5a23-46c2-b735-e7a1ef913743 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:04.218814 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1340/2675] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:04 2022] PUT /identity/v3/projects/209eb7e9dcd94e0aa7c0401e18cfe89c/users/7ac1fcb8d0d142739af91ec8aafdeb23/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:04.229404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf2e9d-259b-4bd7-8006-cd19e1706ef9 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:04.229636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf2e9d-259b-4bd7-8006-cd19e1706ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:04.229954 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf2e9d-259b-4bd7-8006-cd19e1706ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:04.257266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3cdf2e9d-259b-4bd7-8006-cd19e1706ef9 None None] MFA Rules not processed for user `7ac1fcb8d0d142739af91ec8aafdeb23`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:04.258466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3cdf2e9d-259b-4bd7-8006-cd19e1706ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:04.335862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1336/2676] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:04 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:16:04.348068 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4a605-5a23-46c2-b735-e7a1ef913743 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:04.350582 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e5a4a605-5a23-46c2-b735-e7a1ef913743 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:04.380265 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:04.412109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a4a605-5a23-46c2-b735-e7a1ef913743 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:04.416089 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253eb347-a710-4799-9b63-85ff1e010c88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:04.416772 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eb347-a710-4799-9b63-85ff1e010c88 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:04.417055 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eb347-a710-4799-9b63-85ff1e010c88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:04.417375 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253eb347-a710-4799-9b63-85ff1e010c88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:04.426387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253eb347-a710-4799-9b63-85ff1e010c88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:04.427869 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253eb347-a710-4799-9b63-85ff1e010c88 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:04.437128 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1341/2677] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:04 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:04.465422 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf2e9d-259b-4bd7-8006-cd19e1706ef9 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:04.507974 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:04.532550 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf2e9d-259b-4bd7-8006-cd19e1706ef9 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:04.538187 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eaaee8-3367-4db5-8d27-072eac3a4f50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:04.539083 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eaaee8-3367-4db5-8d27-072eac3a4f50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:04.539378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eaaee8-3367-4db5-8d27-072eac3a4f50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:04.539640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74eaaee8-3367-4db5-8d27-072eac3a4f50 admin admin] PATH_INFO: `/v3/users/62c666e5e33e4db38cb82a681f9c7fac` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:04.541106 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eaaee8-3367-4db5-8d27-072eac3a4f50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=62c666e5e33e4db38cb82a681f9c7fac)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:04.543060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74eaaee8-3367-4db5-8d27-072eac3a4f50 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:04.592151 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-74eaaee8-3367-4db5-8d27-072eac3a4f50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '62c666e5e33e4db38cb82a681f9c7fac'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:04.604346 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-74eaaee8-3367-4db5-8d27-072eac3a4f50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '62c666e5e33e4db38cb82a681f9c7fac'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:04.613275 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-74eaaee8-3367-4db5-8d27-072eac3a4f50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '62c666e5e33e4db38cb82a681f9c7fac'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:04.613925 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-74eaaee8-3367-4db5-8d27-072eac3a4f50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '62c666e5e33e4db38cb82a681f9c7fac'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:04.620910 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-74eaaee8-3367-4db5-8d27-072eac3a4f50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '62c666e5e33e4db38cb82a681f9c7fac'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:04.627180 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1337/2678] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:16:04 2022] DELETE /identity/v3/users/62c666e5e33e4db38cb82a681f9c7fac => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:04.632167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2afd5-e645-477a-a61a-7381b642d80d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:04.632487 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2afd5-e645-477a-a61a-7381b642d80d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:04.632798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1d2afd5-e645-477a-a61a-7381b642d80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:04.651973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e1d2afd5-e645-477a-a61a-7381b642d80d None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:04.653361 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e1d2afd5-e645-477a-a61a-7381b642d80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:04.720476 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1342/2679] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:16:04.731658 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eaaee8-3367-4db5-8d27-072eac3a4f50 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:04.733214 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-74eaaee8-3367-4db5-8d27-072eac3a4f50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:04.794012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74eaaee8-3367-4db5-8d27-072eac3a4f50 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:04.797671 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb60307b-3eda-4c8b-913c-1206a1089688 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:04.798326 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60307b-3eda-4c8b-913c-1206a1089688 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:04.798615 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60307b-3eda-4c8b-913c-1206a1089688 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:04.798899 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60307b-3eda-4c8b-913c-1206a1089688 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:04.808222 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb60307b-3eda-4c8b-913c-1206a1089688 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:04.809721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb60307b-3eda-4c8b-913c-1206a1089688 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:04.820039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1338/2680] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:05.105785 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2afd5-e645-477a-a61a-7381b642d80d None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:05.107267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e1d2afd5-e645-477a-a61a-7381b642d80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:05.122174 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1d2afd5-e645-477a-a61a-7381b642d80d None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:05.140712 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250eda7-0111-4f9e-90d0-b1657276f2e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:05.141029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250eda7-0111-4f9e-90d0-b1657276f2e5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:05.141397 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250eda7-0111-4f9e-90d0-b1657276f2e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:05.141743 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6250eda7-0111-4f9e-90d0-b1657276f2e5 admin admin] PATH_INFO: `/v3/projects/72e1d885fefc45b39a6fff0b0c43956f` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:05.143091 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250eda7-0111-4f9e-90d0-b1657276f2e5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=72e1d885fefc45b39a6fff0b0c43956f)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:05.145175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6250eda7-0111-4f9e-90d0-b1657276f2e5 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:05.198215 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6250eda7-0111-4f9e-90d0-b1657276f2e5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '72e1d885fefc45b39a6fff0b0c43956f', 'request_id': 'req-6250eda7-0111-4f9e-90d0-b1657276f2e5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:05.203874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6250eda7-0111-4f9e-90d0-b1657276f2e5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '72e1d885fefc45b39a6fff0b0c43956f', 'request_id': 'req-6250eda7-0111-4f9e-90d0-b1657276f2e5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:05.213367 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6250eda7-0111-4f9e-90d0-b1657276f2e5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '72e1d885fefc45b39a6fff0b0c43956f', 'request_id': 'req-6250eda7-0111-4f9e-90d0-b1657276f2e5'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:05.225268 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6250eda7-0111-4f9e-90d0-b1657276f2e5 admin admin] The token cache is being invalidate because project 72e1d885fefc45b39a6fff0b0c43956f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:05.225573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6250eda7-0111-4f9e-90d0-b1657276f2e5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:05.226230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-6250eda7-0111-4f9e-90d0-b1657276f2e5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:05.227807 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1343/2681] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:05 2022] DELETE /identity/v3/projects/72e1d885fefc45b39a6fff0b0c43956f => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:05.232002 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb60307b-3eda-4c8b-913c-1206a1089688 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:05.242584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb60307b-3eda-4c8b-913c-1206a1089688 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:05.308031 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005f606-188c-4f10-84da-8545f1efb055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:05.308935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005f606-188c-4f10-84da-8545f1efb055 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:05.309216 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005f606-188c-4f10-84da-8545f1efb055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:05.309599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005f606-188c-4f10-84da-8545f1efb055 admin admin] PATH_INFO: `/v3/users/7ac1fcb8d0d142739af91ec8aafdeb23` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:05.310880 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e005f606-188c-4f10-84da-8545f1efb055 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7ac1fcb8d0d142739af91ec8aafdeb23)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:05.312867 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e005f606-188c-4f10-84da-8545f1efb055 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:05.314591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:05.364282 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e005f606-188c-4f10-84da-8545f1efb055 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7ac1fcb8d0d142739af91ec8aafdeb23'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:05.379124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e005f606-188c-4f10-84da-8545f1efb055 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7ac1fcb8d0d142739af91ec8aafdeb23'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:05.388626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e005f606-188c-4f10-84da-8545f1efb055 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7ac1fcb8d0d142739af91ec8aafdeb23'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:05.389263 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e005f606-188c-4f10-84da-8545f1efb055 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7ac1fcb8d0d142739af91ec8aafdeb23'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:05.396469 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e005f606-188c-4f10-84da-8545f1efb055 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7ac1fcb8d0d142739af91ec8aafdeb23'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:05.402807 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1339/2682] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:16:05 2022] DELETE /identity/v3/users/7ac1fcb8d0d142739af91ec8aafdeb23 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:05.680475 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250eda7-0111-4f9e-90d0-b1657276f2e5 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:05.682258 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6250eda7-0111-4f9e-90d0-b1657276f2e5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:05.698020 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6250eda7-0111-4f9e-90d0-b1657276f2e5 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:05.755481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:05.767033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90d3adc0-41f9-46c7-b725-ef55554c53b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:05.768173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3adc0-41f9-46c7-b725-ef55554c53b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:05.768567 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3adc0-41f9-46c7-b725-ef55554c53b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:05.768998 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d3adc0-41f9-46c7-b725-ef55554c53b8 admin admin] PATH_INFO: `/v3/projects/209eb7e9dcd94e0aa7c0401e18cfe89c` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:05.770306 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3adc0-41f9-46c7-b725-ef55554c53b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=209eb7e9dcd94e0aa7c0401e18cfe89c)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:05.772376 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90d3adc0-41f9-46c7-b725-ef55554c53b8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:05.823741 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-90d3adc0-41f9-46c7-b725-ef55554c53b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '209eb7e9dcd94e0aa7c0401e18cfe89c', 'request_id': 'req-90d3adc0-41f9-46c7-b725-ef55554c53b8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:05.829747 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-90d3adc0-41f9-46c7-b725-ef55554c53b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '209eb7e9dcd94e0aa7c0401e18cfe89c', 'request_id': 'req-90d3adc0-41f9-46c7-b725-ef55554c53b8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:05.838479 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-90d3adc0-41f9-46c7-b725-ef55554c53b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '209eb7e9dcd94e0aa7c0401e18cfe89c', 'request_id': 'req-90d3adc0-41f9-46c7-b725-ef55554c53b8'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:05.851468 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-90d3adc0-41f9-46c7-b725-ef55554c53b8 admin admin] The token cache is being invalidate because project 209eb7e9dcd94e0aa7c0401e18cfe89c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:05.851768 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-90d3adc0-41f9-46c7-b725-ef55554c53b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:05.852448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-90d3adc0-41f9-46c7-b725-ef55554c53b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:05.854024 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1344/2683] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:05 2022] DELETE /identity/v3/projects/209eb7e9dcd94e0aa7c0401e18cfe89c => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:16:05.860065 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57553527-310a-4064-b037-5cf50e5a101c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:05.860541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57553527-310a-4064-b037-5cf50e5a101c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:05.860656 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57553527-310a-4064-b037-5cf50e5a101c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:05.878845 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-57553527-310a-4064-b037-5cf50e5a101c None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:05.880076 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-57553527-310a-4064-b037-5cf50e5a101c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:05.949470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1340/2684] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:05.957650 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642bd8c-c904-43d6-ba51-817ec162967b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:05.957940 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642bd8c-c904-43d6-ba51-817ec162967b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:05.958221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e642bd8c-c904-43d6-ba51-817ec162967b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:05.976029 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-e642bd8c-c904-43d6-ba51-817ec162967b None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:05.977223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-e642bd8c-c904-43d6-ba51-817ec162967b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:05.985034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1345/2685] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:05.989248 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57553527-310a-4064-b037-5cf50e5a101c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:05.998891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57553527-310a-4064-b037-5cf50e5a101c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:06.003767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1dfcb-fe84-492b-9a90-1d2a968fd675 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:06.004450 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1dfcb-fe84-492b-9a90-1d2a968fd675 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:06.004736 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1dfcb-fe84-492b-9a90-1d2a968fd675 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:06.005015 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa1dfcb-fe84-492b-9a90-1d2a968fd675 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:06.005744 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1dfcb-fe84-492b-9a90-1d2a968fd675 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:16:06.006056 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1dfcb-fe84-492b-9a90-1d2a968fd675 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:06.007665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa1dfcb-fe84-492b-9a90-1d2a968fd675 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:06.019701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1341/2686] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:16:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:16:06.024283 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642bd8c-c904-43d6-ba51-817ec162967b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:06.035008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e642bd8c-c904-43d6-ba51-817ec162967b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:06.040953 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f475c9-12ec-4366-9c5e-88b7c3a090f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:06.041635 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f475c9-12ec-4366-9c5e-88b7c3a090f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:06.041922 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f475c9-12ec-4366-9c5e-88b7c3a090f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:06.042202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2f475c9-12ec-4366-9c5e-88b7c3a090f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:06.042902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f475c9-12ec-4366-9c5e-88b7c3a090f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:06.044880 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2f475c9-12ec-4366-9c5e-88b7c3a090f7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:06.074923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1346/2687] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:06 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:06.078959 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1dfcb-fe84-492b-9a90-1d2a968fd675 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:06.091228 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa1dfcb-fe84-492b-9a90-1d2a968fd675 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:06.161358 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f65e0-803f-4a58-aeb7-5bdcd621d3b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:06.162264 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f65e0-803f-4a58-aeb7-5bdcd621d3b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:06.162557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f65e0-803f-4a58-aeb7-5bdcd621d3b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:06.162840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc7f65e0-803f-4a58-aeb7-5bdcd621d3b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:06.163555 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f65e0-803f-4a58-aeb7-5bdcd621d3b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:06.165932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc7f65e0-803f-4a58-aeb7-5bdcd621d3b9 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:06.176577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:06.197276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1342/2688] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:06 2022] POST /identity/v3/users => generated 288 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:06.202191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f475c9-12ec-4366-9c5e-88b7c3a090f7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:06.213134 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2f475c9-12ec-4366-9c5e-88b7c3a090f7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:06.218217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586c290-8ff4-4393-8852-dabebcd3817e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:06.218990 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586c290-8ff4-4393-8852-dabebcd3817e admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:06.219282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586c290-8ff4-4393-8852-dabebcd3817e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:06.219571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0586c290-8ff4-4393-8852-dabebcd3817e admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:06.220349 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0586c290-8ff4-4393-8852-dabebcd3817e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:06.222010 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0586c290-8ff4-4393-8852-dabebcd3817e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:06.232723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1347/2689] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:06.236935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f65e0-803f-4a58-aeb7-5bdcd621d3b9 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:06.248038 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc7f65e0-803f-4a58-aeb7-5bdcd621d3b9 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:06.252803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee7d3ec-249d-4490-a9ac-40b6bd09a445 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:06.253477 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7d3ec-249d-4490-a9ac-40b6bd09a445 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:06.253771 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7d3ec-249d-4490-a9ac-40b6bd09a445 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:06.254058 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee7d3ec-249d-4490-a9ac-40b6bd09a445 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:06.254803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7d3ec-249d-4490-a9ac-40b6bd09a445 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:06.256473 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee7d3ec-249d-4490-a9ac-40b6bd09a445 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:06.266809 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1343/2690] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:06.271137 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586c290-8ff4-4393-8852-dabebcd3817e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:06.281753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0586c290-8ff4-4393-8852-dabebcd3817e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:06.287076 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497a48a-716b-4bda-ad3d-49b309117856 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:06.288004 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497a48a-716b-4bda-ad3d-49b309117856 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:06.288297 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497a48a-716b-4bda-ad3d-49b309117856 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:06.288579 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7497a48a-716b-4bda-ad3d-49b309117856 admin admin] PATH_INFO: `/v3/projects/7b3ab2fac8024c19abc57298984fd12a/users/576375cde39245cb89ab7f9ef8928270/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:06.305146 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7497a48a-716b-4bda-ad3d-49b309117856 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7b3ab2fac8024c19abc57298984fd12a, user_id=576375cde39245cb89ab7f9ef8928270, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:06.307860 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7497a48a-716b-4bda-ad3d-49b309117856 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:06.323532 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1348/2691] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:06 2022] PUT /identity/v3/projects/7b3ab2fac8024c19abc57298984fd12a/users/576375cde39245cb89ab7f9ef8928270/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:06.334998 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a8ba0-868f-418d-8125-cdee8015b835 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:06.335231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a8ba0-868f-418d-8125-cdee8015b835 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:06.335523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487a8ba0-868f-418d-8125-cdee8015b835 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:06.368314 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-487a8ba0-868f-418d-8125-cdee8015b835 None None] MFA Rules not processed for user `576375cde39245cb89ab7f9ef8928270`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:06.369557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-487a8ba0-868f-418d-8125-cdee8015b835 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:06.440066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1344/2692] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:06 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:06.444834 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497a48a-716b-4bda-ad3d-49b309117856 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:06.511607 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7497a48a-716b-4bda-ad3d-49b309117856 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:06.516431 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91d261-a3f1-4b10-a95f-d6e4ef835bcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:06.517103 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91d261-a3f1-4b10-a95f-d6e4ef835bcf admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:06.517397 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91d261-a3f1-4b10-a95f-d6e4ef835bcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:06.517678 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b91d261-a3f1-4b10-a95f-d6e4ef835bcf admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:06.518386 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91d261-a3f1-4b10-a95f-d6e4ef835bcf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:06.520327 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b91d261-a3f1-4b10-a95f-d6e4ef835bcf admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:06.546473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1349/2693] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:06 2022] POST /identity/v3/projects => generated 373 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:06.550475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a8ba0-868f-418d-8125-cdee8015b835 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:06.561470 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487a8ba0-868f-418d-8125-cdee8015b835 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:06.618841 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7692ef-acf4-4062-893f-33a714e430b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:06.619615 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7692ef-acf4-4062-893f-33a714e430b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:06.619948 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7692ef-acf4-4062-893f-33a714e430b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:06.620233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df7692ef-acf4-4062-893f-33a714e430b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:06.620949 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7692ef-acf4-4062-893f-33a714e430b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:06.623080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df7692ef-acf4-4062-893f-33a714e430b1 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:06.643382 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1345/2694] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:06 2022] POST /identity/v3/users => generated 293 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:06.647999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91d261-a3f1-4b10-a95f-d6e4ef835bcf admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:06.653479 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:06.659778 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b91d261-a3f1-4b10-a95f-d6e4ef835bcf admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:06.665728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99171ebe-1cd5-47f6-b35b-a6c498fb1b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:06.666622 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99171ebe-1cd5-47f6-b35b-a6c498fb1b3d admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:06.667090 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99171ebe-1cd5-47f6-b35b-a6c498fb1b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:06.667387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99171ebe-1cd5-47f6-b35b-a6c498fb1b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:06.668369 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99171ebe-1cd5-47f6-b35b-a6c498fb1b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:06.670294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99171ebe-1cd5-47f6-b35b-a6c498fb1b3d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:06.680955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1350/2695] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:06.685311 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7692ef-acf4-4062-893f-33a714e430b1 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:06.697300 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df7692ef-acf4-4062-893f-33a714e430b1 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:06.704061 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b5d6e-c360-4822-b319-cd1886a2b2a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:06.705275 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b5d6e-c360-4822-b319-cd1886a2b2a7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:06.705852 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b5d6e-c360-4822-b319-cd1886a2b2a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:06.706200 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b2b5d6e-c360-4822-b319-cd1886a2b2a7 admin admin] PATH_INFO: `/v3/projects/5c10363f62a04717a7a3f9c3f1862ed6/users/da23817503854d668de4ac813ad99473/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:06.723548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b5d6e-c360-4822-b319-cd1886a2b2a7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c10363f62a04717a7a3f9c3f1862ed6, user_id=da23817503854d668de4ac813ad99473, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:06.726663 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b2b5d6e-c360-4822-b319-cd1886a2b2a7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:06.740917 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1346/2696] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:06 2022] PUT /identity/v3/projects/5c10363f62a04717a7a3f9c3f1862ed6/users/da23817503854d668de4ac813ad99473/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:06.744982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99171ebe-1cd5-47f6-b35b-a6c498fb1b3d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:06.755514 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99171ebe-1cd5-47f6-b35b-a6c498fb1b3d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:06.817871 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147ea7c-3ef7-4996-b11c-9bb158cd0be4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:06.818586 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147ea7c-3ef7-4996-b11c-9bb158cd0be4 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:06.818877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147ea7c-3ef7-4996-b11c-9bb158cd0be4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:06.819159 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147ea7c-3ef7-4996-b11c-9bb158cd0be4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:06.819919 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a147ea7c-3ef7-4996-b11c-9bb158cd0be4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:06.821549 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a147ea7c-3ef7-4996-b11c-9bb158cd0be4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:06.832077 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1351/2697] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:06 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:16:06.836702 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b5d6e-c360-4822-b319-cd1886a2b2a7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:06.847370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b2b5d6e-c360-4822-b319-cd1886a2b2a7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:06.852272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e594e3-1957-4451-8e1c-368701577d59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:06.853172 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e594e3-1957-4451-8e1c-368701577d59 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:06.853462 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e594e3-1957-4451-8e1c-368701577d59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:06.853772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e594e3-1957-4451-8e1c-368701577d59 admin admin] PATH_INFO: `/v3/projects/5c10363f62a04717a7a3f9c3f1862ed6/users/da23817503854d668de4ac813ad99473/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:06.855855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e594e3-1957-4451-8e1c-368701577d59 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5c10363f62a04717a7a3f9c3f1862ed6, user_id=da23817503854d668de4ac813ad99473, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:06.858718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e594e3-1957-4451-8e1c-368701577d59 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:06.869517 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1347/2698] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:06 2022] PUT /identity/v3/projects/5c10363f62a04717a7a3f9c3f1862ed6/users/da23817503854d668de4ac813ad99473/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:06.881175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde53049-64e4-4072-bb90-cbce06fcf261 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:06.881552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde53049-64e4-4072-bb90-cbce06fcf261 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:06.881959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dde53049-64e4-4072-bb90-cbce06fcf261 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:06.919158 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-dde53049-64e4-4072-bb90-cbce06fcf261 None None] MFA Rules not processed for user `da23817503854d668de4ac813ad99473`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:06.920466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-dde53049-64e4-4072-bb90-cbce06fcf261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:06.986998 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e594e3-1957-4451-8e1c-368701577d59 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:06.988870 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-17e594e3-1957-4451-8e1c-368701577d59 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:06.995303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1352/2699] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:06 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:07.006906 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde53049-64e4-4072-bb90-cbce06fcf261 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:07.008372 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-dde53049-64e4-4072-bb90-cbce06fcf261 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:07.022615 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:07.061181 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e594e3-1957-4451-8e1c-368701577d59 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:07.066857 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c74ed7-d803-40e6-870a-92000f2f2f4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:07.067763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c74ed7-d803-40e6-870a-92000f2f2f4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:07.068131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c74ed7-d803-40e6-870a-92000f2f2f4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:07.068416 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c74ed7-d803-40e6-870a-92000f2f2f4b admin admin] PATH_INFO: `/v3/users/cb701ce5490d46bcb2b36592167289fb` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:07.069625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c74ed7-d803-40e6-870a-92000f2f2f4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cb701ce5490d46bcb2b36592167289fb)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:07.070945 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dde53049-64e4-4072-bb90-cbce06fcf261 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:07.071613 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c74ed7-d803-40e6-870a-92000f2f2f4b admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:07.074653 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653f88b6-d09f-4cb6-ba82-98d455a11dc7 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:07.075315 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f88b6-d09f-4cb6-ba82-98d455a11dc7 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:07.075636 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f88b6-d09f-4cb6-ba82-98d455a11dc7 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:07.075949 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653f88b6-d09f-4cb6-ba82-98d455a11dc7 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:07.103218 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:07.121073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f4c74ed7-d803-40e6-870a-92000f2f2f4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cb701ce5490d46bcb2b36592167289fb'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:07.133607 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f4c74ed7-d803-40e6-870a-92000f2f2f4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cb701ce5490d46bcb2b36592167289fb'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:07.135952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f88b6-d09f-4cb6-ba82-98d455a11dc7 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:07.137608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653f88b6-d09f-4cb6-ba82-98d455a11dc7 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:07.141682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f4c74ed7-d803-40e6-870a-92000f2f2f4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cb701ce5490d46bcb2b36592167289fb'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:07.142483 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f4c74ed7-d803-40e6-870a-92000f2f2f4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cb701ce5490d46bcb2b36592167289fb'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:07.147447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1353/2700] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:07 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:07.150168 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-f4c74ed7-d803-40e6-870a-92000f2f2f4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cb701ce5490d46bcb2b36592167289fb'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:07.157309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1348/2701] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:16:06 2022] DELETE /identity/v3/users/cb701ce5490d46bcb2b36592167289fb => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:16:07.161922 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec5f84-f77b-4fd9-be7d-1b204a584bc4 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:07.162221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec5f84-f77b-4fd9-be7d-1b204a584bc4 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:07.162499 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ec5f84-f77b-4fd9-be7d-1b204a584bc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:07.183013 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-a8ec5f84-f77b-4fd9-be7d-1b204a584bc4 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:07.184376 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a8ec5f84-f77b-4fd9-be7d-1b204a584bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:07.248622 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1354/2702] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:07.258753 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c74ed7-d803-40e6-870a-92000f2f2f4b admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:07.260399 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f4c74ed7-d803-40e6-870a-92000f2f2f4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:07.326091 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c74ed7-d803-40e6-870a-92000f2f2f4b admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:07.330223 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dddcf-ad54-473f-9e72-b083157907b2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:07.331075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dddcf-ad54-473f-9e72-b083157907b2 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:07.331384 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dddcf-ad54-473f-9e72-b083157907b2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:07.331672 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898dddcf-ad54-473f-9e72-b083157907b2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:07.341501 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898dddcf-ad54-473f-9e72-b083157907b2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:07.342976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898dddcf-ad54-473f-9e72-b083157907b2 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:07.353464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1349/2703] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:07.556443 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec5f84-f77b-4fd9-be7d-1b204a584bc4 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:07.557934 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a8ec5f84-f77b-4fd9-be7d-1b204a584bc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:07.570753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ec5f84-f77b-4fd9-be7d-1b204a584bc4 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:07.575530 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd18a20-d1af-4456-83c6-4a5e91433884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:07.576458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd18a20-d1af-4456-83c6-4a5e91433884 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:07.576754 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd18a20-d1af-4456-83c6-4a5e91433884 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:07.577039 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd18a20-d1af-4456-83c6-4a5e91433884 admin admin] PATH_INFO: `/v3/users/81751664267043eab0c71bb0b2e95692` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:07.578285 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd18a20-d1af-4456-83c6-4a5e91433884 admin admin] RBAC: Authorizing `identity:delete_user(user_id=81751664267043eab0c71bb0b2e95692)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:07.580291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd18a20-d1af-4456-83c6-4a5e91433884 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:07.631244 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cdd18a20-d1af-4456-83c6-4a5e91433884 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '81751664267043eab0c71bb0b2e95692'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:07.632008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cdd18a20-d1af-4456-83c6-4a5e91433884 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '81751664267043eab0c71bb0b2e95692'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:07.643131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dddcf-ad54-473f-9e72-b083157907b2 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:07.644708 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-898dddcf-ad54-473f-9e72-b083157907b2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:07.651287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cdd18a20-d1af-4456-83c6-4a5e91433884 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '81751664267043eab0c71bb0b2e95692'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:07.661661 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898dddcf-ad54-473f-9e72-b083157907b2 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:07.662843 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cdd18a20-d1af-4456-83c6-4a5e91433884 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '81751664267043eab0c71bb0b2e95692'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:07.663616 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cdd18a20-d1af-4456-83c6-4a5e91433884 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '81751664267043eab0c71bb0b2e95692'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:07.667277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95733ae8-30b1-4fce-a6c7-7ce14abaf1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:07.669094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95733ae8-30b1-4fce-a6c7-7ce14abaf1cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:07.669387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95733ae8-30b1-4fce-a6c7-7ce14abaf1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:07.669680 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95733ae8-30b1-4fce-a6c7-7ce14abaf1cf admin admin] PATH_INFO: `/v3/projects/1641c9d598d04c00aeff2591c378f784` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:07.671220 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95733ae8-30b1-4fce-a6c7-7ce14abaf1cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=1641c9d598d04c00aeff2591c378f784)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:07.673058 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95733ae8-30b1-4fce-a6c7-7ce14abaf1cf admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:07.679223 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1355/2704] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:16:07 2022] DELETE /identity/v3/users/81751664267043eab0c71bb0b2e95692 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:07.683788 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1721d82-40ac-4194-93d1-9b40359d13f7 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:07.684394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1721d82-40ac-4194-93d1-9b40359d13f7 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:07.684394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1721d82-40ac-4194-93d1-9b40359d13f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:07.703254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c1721d82-40ac-4194-93d1-9b40359d13f7 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:07.704545 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c1721d82-40ac-4194-93d1-9b40359d13f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:07.722274 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:07.726410 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-95733ae8-30b1-4fce-a6c7-7ce14abaf1cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1641c9d598d04c00aeff2591c378f784', 'request_id': 'req-95733ae8-30b1-4fce-a6c7-7ce14abaf1cf'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:07.737060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-95733ae8-30b1-4fce-a6c7-7ce14abaf1cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1641c9d598d04c00aeff2591c378f784', 'request_id': 'req-95733ae8-30b1-4fce-a6c7-7ce14abaf1cf'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:07.740386 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:07.743945 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-95733ae8-30b1-4fce-a6c7-7ce14abaf1cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1641c9d598d04c00aeff2591c378f784', 'request_id': 'req-95733ae8-30b1-4fce-a6c7-7ce14abaf1cf'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:07.756101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-95733ae8-30b1-4fce-a6c7-7ce14abaf1cf admin admin] The token cache is being invalidate because project 1641c9d598d04c00aeff2591c378f784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:07.756396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-95733ae8-30b1-4fce-a6c7-7ce14abaf1cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:07.757055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-95733ae8-30b1-4fce-a6c7-7ce14abaf1cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:07.758833 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1350/2705] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:07 2022] DELETE /identity/v3/projects/1641c9d598d04c00aeff2591c378f784 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:07.763447 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935758a-6d8c-4284-9c12-5810f6c8e82c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:07.763973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935758a-6d8c-4284-9c12-5810f6c8e82c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:07.764209 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f935758a-6d8c-4284-9c12-5810f6c8e82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:07.768390 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1356/2706] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:16:07.778877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1721d82-40ac-4194-93d1-9b40359d13f7 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:07.780261 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c1721d82-40ac-4194-93d1-9b40359d13f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:07.783137 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-f935758a-6d8c-4284-9c12-5810f6c8e82c None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:07.784372 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f935758a-6d8c-4284-9c12-5810f6c8e82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:07.837996 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1721d82-40ac-4194-93d1-9b40359d13f7 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:07.844697 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1351/2707] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:16:07.852477 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f5abc-d792-4a1f-8128-4fb43667a8b4 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:07.852772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f5abc-d792-4a1f-8128-4fb43667a8b4 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:07.853053 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5f5abc-d792-4a1f-8128-4fb43667a8b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:07.870940 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1e5f5abc-d792-4a1f-8128-4fb43667a8b4 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:07.872184 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1e5f5abc-d792-4a1f-8128-4fb43667a8b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:07.880452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1352/2708] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:16:07.884526 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f5abc-d792-4a1f-8128-4fb43667a8b4 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:07.891675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e32ac-2cd1-41c7-b835-86e41b4fb99f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:07.892338 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e32ac-2cd1-41c7-b835-86e41b4fb99f service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:07.892628 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e32ac-2cd1-41c7-b835-86e41b4fb99f service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:07.892908 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b8e32ac-2cd1-41c7-b835-86e41b4fb99f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:07.893575 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e5f5abc-d792-4a1f-8128-4fb43667a8b4 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:07.898185 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0648f-103f-4c30-ae40-2f836317fe34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:07.898830 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0648f-103f-4c30-ae40-2f836317fe34 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:07.899125 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0648f-103f-4c30-ae40-2f836317fe34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:07.899409 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c0648f-103f-4c30-ae40-2f836317fe34 admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:07.900249 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0648f-103f-4c30-ae40-2f836317fe34 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:16:07.900565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0648f-103f-4c30-ae40-2f836317fe34 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:07.902096 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e32ac-2cd1-41c7-b835-86e41b4fb99f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:07.902210 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c0648f-103f-4c30-ae40-2f836317fe34 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:07.903541 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b8e32ac-2cd1-41c7-b835-86e41b4fb99f service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:07.912964 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1357/2709] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:16:07.914190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1353/2710] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:16:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:16:07.918497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e32ac-2cd1-41c7-b835-86e41b4fb99f service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:07.928174 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b8e32ac-2cd1-41c7-b835-86e41b4fb99f service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:07.933008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4f0a6-43e4-4646-975d-dfbc0d457ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:07.933679 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4f0a6-43e4-4646-975d-dfbc0d457ecf admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:07.933968 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4f0a6-43e4-4646-975d-dfbc0d457ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:07.934249 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f4f0a6-43e4-4646-975d-dfbc0d457ecf admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:07.934982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4f0a6-43e4-4646-975d-dfbc0d457ecf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:07.937005 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98f4f0a6-43e4-4646-975d-dfbc0d457ecf admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:07.964706 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1358/2711] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:07 2022] POST /identity/v3/projects => generated 371 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:16:07.969121 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0648f-103f-4c30-ae40-2f836317fe34 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:07.980179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c0648f-103f-4c30-ae40-2f836317fe34 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:08.041033 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174269d-71d5-40f8-b924-0fc4d85215c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:08.041716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174269d-71d5-40f8-b924-0fc4d85215c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:08.042004 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174269d-71d5-40f8-b924-0fc4d85215c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:08.042289 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f174269d-71d5-40f8-b924-0fc4d85215c7 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:08.042990 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174269d-71d5-40f8-b924-0fc4d85215c7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:08.045000 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f174269d-71d5-40f8-b924-0fc4d85215c7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:08.068330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1354/2712] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:07 2022] POST /identity/v3/users => generated 286 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:08.072723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4f0a6-43e4-4646-975d-dfbc0d457ecf admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:08.084503 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f4f0a6-43e4-4646-975d-dfbc0d457ecf admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:08.089170 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e525197-8187-4e80-9438-18b984b40ccf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:08.089852 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e525197-8187-4e80-9438-18b984b40ccf admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:08.090140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e525197-8187-4e80-9438-18b984b40ccf admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:08.090426 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e525197-8187-4e80-9438-18b984b40ccf admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:08.091169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e525197-8187-4e80-9438-18b984b40ccf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:08.092915 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e525197-8187-4e80-9438-18b984b40ccf admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:08.104149 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1359/2713] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:08.108052 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174269d-71d5-40f8-b924-0fc4d85215c7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:08.118498 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f174269d-71d5-40f8-b924-0fc4d85215c7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:08.123761 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793d960-c7c8-4fce-864b-a4c9c3b11e06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:08.124509 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793d960-c7c8-4fce-864b-a4c9c3b11e06 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:08.124786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793d960-c7c8-4fce-864b-a4c9c3b11e06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:08.125065 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c793d960-c7c8-4fce-864b-a4c9c3b11e06 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:08.125800 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793d960-c7c8-4fce-864b-a4c9c3b11e06 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:08.127420 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c793d960-c7c8-4fce-864b-a4c9c3b11e06 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:08.139235 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1355/2714] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:08 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:08.143481 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e525197-8187-4e80-9438-18b984b40ccf admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:08.154289 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e525197-8187-4e80-9438-18b984b40ccf admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:08.159226 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625c32a-387c-4e5f-98b6-44266e3fa09e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:08.160277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625c32a-387c-4e5f-98b6-44266e3fa09e admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:08.160571 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625c32a-387c-4e5f-98b6-44266e3fa09e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:08.160909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625c32a-387c-4e5f-98b6-44266e3fa09e admin admin] PATH_INFO: `/v3/projects/c78164ecb9964a099322629d8df3d73a/users/2ae3536df90b4fa1aac27938a4fd671f/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:08.177248 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625c32a-387c-4e5f-98b6-44266e3fa09e admin admin] RBAC: Authorizing `identity:create_grant(project_id=c78164ecb9964a099322629d8df3d73a, user_id=2ae3536df90b4fa1aac27938a4fd671f, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:08.180141 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f625c32a-387c-4e5f-98b6-44266e3fa09e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:08.185999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793d960-c7c8-4fce-864b-a4c9c3b11e06 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:08.188490 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c793d960-c7c8-4fce-864b-a4c9c3b11e06 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:08.198026 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1360/2715] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:08 2022] PUT /identity/v3/projects/c78164ecb9964a099322629d8df3d73a/users/2ae3536df90b4fa1aac27938a4fd671f/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:16:08.208815 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dec38-1eb3-4aa5-a7f9-c89620b5dee2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:08.209146 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dec38-1eb3-4aa5-a7f9-c89620b5dee2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:08.209434 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e0dec38-1eb3-4aa5-a7f9-c89620b5dee2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:08.223412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c793d960-c7c8-4fce-864b-a4c9c3b11e06 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:08.238660 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4e0dec38-1eb3-4aa5-a7f9-c89620b5dee2 None None] MFA Rules not processed for user `2ae3536df90b4fa1aac27938a4fd671f`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:08.239960 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4e0dec38-1eb3-4aa5-a7f9-c89620b5dee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:08.253623 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:08.272270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:08.285124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fa96f-5810-466a-8a6b-e66d9b2b2617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:08.286018 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa96f-5810-466a-8a6b-e66d9b2b2617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:08.286306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa96f-5810-466a-8a6b-e66d9b2b2617 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:08.286590 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99fa96f-5810-466a-8a6b-e66d9b2b2617 admin admin] PATH_INFO: `/v3/projects/b0f06a5d4c67452dbd4c860175726d4f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:08.287772 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fa96f-5810-466a-8a6b-e66d9b2b2617 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b0f06a5d4c67452dbd4c860175726d4f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:08.289732 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99fa96f-5810-466a-8a6b-e66d9b2b2617 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:08.308717 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1361/2716] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:08 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:08.320300 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0dec38-1eb3-4aa5-a7f9-c89620b5dee2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:08.321671 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4e0dec38-1eb3-4aa5-a7f9-c89620b5dee2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:08.338133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e99fa96f-5810-466a-8a6b-e66d9b2b2617 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b0f06a5d4c67452dbd4c860175726d4f', 'request_id': 'req-e99fa96f-5810-466a-8a6b-e66d9b2b2617'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.346401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e99fa96f-5810-466a-8a6b-e66d9b2b2617 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b0f06a5d4c67452dbd4c860175726d4f', 'request_id': 'req-e99fa96f-5810-466a-8a6b-e66d9b2b2617'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.353142 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e99fa96f-5810-466a-8a6b-e66d9b2b2617 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b0f06a5d4c67452dbd4c860175726d4f', 'request_id': 'req-e99fa96f-5810-466a-8a6b-e66d9b2b2617'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.363505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e99fa96f-5810-466a-8a6b-e66d9b2b2617 admin admin] The token cache is being invalidate because project b0f06a5d4c67452dbd4c860175726d4f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:08.363811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e99fa96f-5810-466a-8a6b-e66d9b2b2617 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.364513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-e99fa96f-5810-466a-8a6b-e66d9b2b2617 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.366174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1356/2717] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:08 2022] DELETE /identity/v3/projects/b0f06a5d4c67452dbd4c860175726d4f => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:08.370064 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fa96f-5810-466a-8a6b-e66d9b2b2617 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:08.371415 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e99fa96f-5810-466a-8a6b-e66d9b2b2617 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:08.380936 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e0dec38-1eb3-4aa5-a7f9-c89620b5dee2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:08.430887 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd95e66-4129-4038-8d51-60b3a36884f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:08.431465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd95e66-4129-4038-8d51-60b3a36884f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:08.431759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd95e66-4129-4038-8d51-60b3a36884f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:08.432072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd95e66-4129-4038-8d51-60b3a36884f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:08.433998 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ccd95e66-4129-4038-8d51-60b3a36884f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:08.436610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99fa96f-5810-466a-8a6b-e66d9b2b2617 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:08.442415 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cfb300f-efca-47b5-b66d-3c6711da676d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:08.442415 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb300f-efca-47b5-b66d-3c6711da676d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:08.442667 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb300f-efca-47b5-b66d-3c6711da676d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:08.442803 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cfb300f-efca-47b5-b66d-3c6711da676d admin admin] PATH_INFO: `/v3/users/f3ead1a34f94452197fd51401b7f2b64` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:08.444032 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb300f-efca-47b5-b66d-3c6711da676d admin admin] RBAC: Authorizing `identity:delete_user(user_id=f3ead1a34f94452197fd51401b7f2b64)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:08.445988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cfb300f-efca-47b5-b66d-3c6711da676d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:08.496567 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1cfb300f-efca-47b5-b66d-3c6711da676d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f3ead1a34f94452197fd51401b7f2b64'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.497211 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd95e66-4129-4038-8d51-60b3a36884f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:08.498813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd95e66-4129-4038-8d51-60b3a36884f5 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:08.508527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1362/2718] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:08 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 189 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:08.512068 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1cfb300f-efca-47b5-b66d-3c6711da676d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f3ead1a34f94452197fd51401b7f2b64'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.516257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=83,keystone:DELETE=16,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:08.523831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1cfb300f-efca-47b5-b66d-3c6711da676d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f3ead1a34f94452197fd51401b7f2b64'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.524654 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1cfb300f-efca-47b5-b66d-3c6711da676d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f3ead1a34f94452197fd51401b7f2b64'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.537599 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd95e66-4129-4038-8d51-60b3a36884f5 service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:08.539181 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ccd95e66-4129-4038-8d51-60b3a36884f5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:08.541314 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1cfb300f-efca-47b5-b66d-3c6711da676d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f3ead1a34f94452197fd51401b7f2b64'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.549008 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1357/2719] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:16:08 2022] DELETE /identity/v3/users/f3ead1a34f94452197fd51401b7f2b64 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:16:08.554966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd95e66-4129-4038-8d51-60b3a36884f5 service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:08.604053 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:08.627615 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3089848-72ac-4894-be12-8ed5384e7c34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:08.628439 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3089848-72ac-4894-be12-8ed5384e7c34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:08.628777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3089848-72ac-4894-be12-8ed5384e7c34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:08.629002 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3089848-72ac-4894-be12-8ed5384e7c34 admin admin] PATH_INFO: `/v3/users/2ae3536df90b4fa1aac27938a4fd671f` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:08.630561 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3089848-72ac-4894-be12-8ed5384e7c34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2ae3536df90b4fa1aac27938a4fd671f)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:08.632734 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3089848-72ac-4894-be12-8ed5384e7c34 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:08.689726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d3089848-72ac-4894-be12-8ed5384e7c34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2ae3536df90b4fa1aac27938a4fd671f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.690423 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d3089848-72ac-4894-be12-8ed5384e7c34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2ae3536df90b4fa1aac27938a4fd671f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.704900 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d3089848-72ac-4894-be12-8ed5384e7c34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2ae3536df90b4fa1aac27938a4fd671f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.714865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d3089848-72ac-4894-be12-8ed5384e7c34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2ae3536df90b4fa1aac27938a4fd671f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.715623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-d3089848-72ac-4894-be12-8ed5384e7c34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2ae3536df90b4fa1aac27938a4fd671f'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.729733 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1363/2720] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:16:08 2022] DELETE /identity/v3/users/2ae3536df90b4fa1aac27938a4fd671f => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:16:08.734110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa382f-17d4-477e-a991-1c1e4477ef6c None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:08.734412 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa382f-17d4-477e-a991-1c1e4477ef6c None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:08.734701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5fa382f-17d4-477e-a991-1c1e4477ef6c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:08.752073 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e5fa382f-17d4-477e-a991-1c1e4477ef6c None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:08.753784 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e5fa382f-17d4-477e-a991-1c1e4477ef6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:08.831174 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1358/2721] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:08.841813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3089848-72ac-4894-be12-8ed5384e7c34 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:08.843465 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d3089848-72ac-4894-be12-8ed5384e7c34 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:08.864645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa382f-17d4-477e-a991-1c1e4477ef6c None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:08.866201 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e5fa382f-17d4-477e-a991-1c1e4477ef6c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:08.881640 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5fa382f-17d4-477e-a991-1c1e4477ef6c None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:08.888991 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-651b7327-4811-4b35-8447-e04537570800 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:08.890100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b7327-4811-4b35-8447-e04537570800 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:08.890364 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b7327-4811-4b35-8447-e04537570800 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:08.890651 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-651b7327-4811-4b35-8447-e04537570800 admin admin] PATH_INFO: `/v3/projects/017fe8e6b6a14532938922a038574b0b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:08.899495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b7327-4811-4b35-8447-e04537570800 admin admin] RBAC: Authorizing `identity:delete_project(project_id=017fe8e6b6a14532938922a038574b0b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:08.901616 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-651b7327-4811-4b35-8447-e04537570800 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:08.914496 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3089848-72ac-4894-be12-8ed5384e7c34 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:08.918605 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2d802f-78d2-4d08-bbc7-b3e21c5345ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:08.919604 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d802f-78d2-4d08-bbc7-b3e21c5345ed service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:08.920169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d802f-78d2-4d08-bbc7-b3e21c5345ed service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:08.920677 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2d802f-78d2-4d08-bbc7-b3e21c5345ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:08.932590 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d802f-78d2-4d08-bbc7-b3e21c5345ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:08.934102 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2d802f-78d2-4d08-bbc7-b3e21c5345ed service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:08.949870 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1364/2722] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:08.963766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-651b7327-4811-4b35-8447-e04537570800 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '017fe8e6b6a14532938922a038574b0b', 'request_id': 'req-651b7327-4811-4b35-8447-e04537570800'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.974101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-651b7327-4811-4b35-8447-e04537570800 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '017fe8e6b6a14532938922a038574b0b', 'request_id': 'req-651b7327-4811-4b35-8447-e04537570800'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.980973 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-651b7327-4811-4b35-8447-e04537570800 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '017fe8e6b6a14532938922a038574b0b', 'request_id': 'req-651b7327-4811-4b35-8447-e04537570800'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.992978 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-651b7327-4811-4b35-8447-e04537570800 admin admin] The token cache is being invalidate because project 017fe8e6b6a14532938922a038574b0b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:08.993279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-651b7327-4811-4b35-8447-e04537570800 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.993982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-651b7327-4811-4b35-8447-e04537570800 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:08.995611 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1359/2723] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:08 2022] DELETE /identity/v3/projects/017fe8e6b6a14532938922a038574b0b => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:16:09.001129 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db291ed8-39d5-4cae-a33c-0653f9f6ce0a None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.001277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db291ed8-39d5-4cae-a33c-0653f9f6ce0a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.001447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db291ed8-39d5-4cae-a33c-0653f9f6ce0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.023562 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-db291ed8-39d5-4cae-a33c-0653f9f6ce0a None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:09.024998 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-db291ed8-39d5-4cae-a33c-0653f9f6ce0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:09.087365 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1365/2724] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:09.097042 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed02d7f-5eac-41e0-9036-cc048f7cd322 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.097390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed02d7f-5eac-41e0-9036-cc048f7cd322 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.097637 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed02d7f-5eac-41e0-9036-cc048f7cd322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.116511 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-7ed02d7f-5eac-41e0-9036-cc048f7cd322 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:09.117746 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7ed02d7f-5eac-41e0-9036-cc048f7cd322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:09.126332 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1360/2725] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:09.131310 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db291ed8-39d5-4cae-a33c-0653f9f6ce0a None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.137999 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:09.142306 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db291ed8-39d5-4cae-a33c-0653f9f6ce0a None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.148917 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523d6a5-c3f3-42fa-b855-9cac56e64b28 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.149663 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523d6a5-c3f3-42fa-b855-9cac56e64b28 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.150117 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523d6a5-c3f3-42fa-b855-9cac56e64b28 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.150601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6523d6a5-c3f3-42fa-b855-9cac56e64b28 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.151528 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523d6a5-c3f3-42fa-b855-9cac56e64b28 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:16:09.151960 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523d6a5-c3f3-42fa-b855-9cac56e64b28 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:09.153666 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6523d6a5-c3f3-42fa-b855-9cac56e64b28 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:09.165643 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1366/2726] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:16:09.170232 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed02d7f-5eac-41e0-9036-cc048f7cd322 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.179505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed02d7f-5eac-41e0-9036-cc048f7cd322 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.185480 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b5cf2-d7ee-442c-b8fa-ccf4e54c6dd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.186096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b5cf2-d7ee-442c-b8fa-ccf4e54c6dd8 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.186425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b5cf2-d7ee-442c-b8fa-ccf4e54c6dd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.186724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-605b5cf2-d7ee-442c-b8fa-ccf4e54c6dd8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.187445 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b5cf2-d7ee-442c-b8fa-ccf4e54c6dd8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:09.189439 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-605b5cf2-d7ee-442c-b8fa-ccf4e54c6dd8 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:09.217359 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1361/2727] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:09 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:09.221907 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523d6a5-c3f3-42fa-b855-9cac56e64b28 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.233466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6523d6a5-c3f3-42fa-b855-9cac56e64b28 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.233656 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b5cf2-d7ee-442c-b8fa-ccf4e54c6dd8 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.235498 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-605b5cf2-d7ee-442c-b8fa-ccf4e54c6dd8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:09.282172 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:09.296343 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef2eb5-4a2f-41e7-b301-952a2ac19967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.297154 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef2eb5-4a2f-41e7-b301-952a2ac19967 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.297588 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef2eb5-4a2f-41e7-b301-952a2ac19967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.297921 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ef2eb5-4a2f-41e7-b301-952a2ac19967 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.298952 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef2eb5-4a2f-41e7-b301-952a2ac19967 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:09.301046 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ef2eb5-4a2f-41e7-b301-952a2ac19967 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:09.309407 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-605b5cf2-d7ee-442c-b8fa-ccf4e54c6dd8 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.314058 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb899b9-0091-45a8-b447-d49158fbd7a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.314932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb899b9-0091-45a8-b447-d49158fbd7a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.315220 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb899b9-0091-45a8-b447-d49158fbd7a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.315558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb899b9-0091-45a8-b447-d49158fbd7a0 admin admin] PATH_INFO: `/v3/projects/c78164ecb9964a099322629d8df3d73a` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.316752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb899b9-0091-45a8-b447-d49158fbd7a0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c78164ecb9964a099322629d8df3d73a)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:09.318824 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb899b9-0091-45a8-b447-d49158fbd7a0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:09.323120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1367/2728] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:09 2022] POST /identity/v3/users => generated 294 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:09.327346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef2eb5-4a2f-41e7-b301-952a2ac19967 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.336970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ef2eb5-4a2f-41e7-b301-952a2ac19967 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.341606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efebfba-3900-467e-9076-ccf7598bd835 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.342319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efebfba-3900-467e-9076-ccf7598bd835 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.342611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efebfba-3900-467e-9076-ccf7598bd835 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.342901 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5efebfba-3900-467e-9076-ccf7598bd835 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.343658 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efebfba-3900-467e-9076-ccf7598bd835 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:09.345448 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5efebfba-3900-467e-9076-ccf7598bd835 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:09.359548 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1368/2729] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:16:09.360631 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efebfba-3900-467e-9076-ccf7598bd835 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.369268 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3fb899b9-0091-45a8-b447-d49158fbd7a0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c78164ecb9964a099322629d8df3d73a', 'request_id': 'req-3fb899b9-0091-45a8-b447-d49158fbd7a0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:09.370329 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5efebfba-3900-467e-9076-ccf7598bd835 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.378421 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3fb899b9-0091-45a8-b447-d49158fbd7a0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c78164ecb9964a099322629d8df3d73a', 'request_id': 'req-3fb899b9-0091-45a8-b447-d49158fbd7a0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:09.385926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3fb899b9-0091-45a8-b447-d49158fbd7a0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c78164ecb9964a099322629d8df3d73a', 'request_id': 'req-3fb899b9-0091-45a8-b447-d49158fbd7a0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:09.396909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3fb899b9-0091-45a8-b447-d49158fbd7a0 admin admin] The token cache is being invalidate because project c78164ecb9964a099322629d8df3d73a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:09.397217 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3fb899b9-0091-45a8-b447-d49158fbd7a0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:09.398037 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3fb899b9-0091-45a8-b447-d49158fbd7a0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:09.399748 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1362/2730] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:09 2022] DELETE /identity/v3/projects/c78164ecb9964a099322629d8df3d73a => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:16:09.405272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb8526-66f3-4822-9944-9d17828c4322 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.405272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb8526-66f3-4822-9944-9d17828c4322 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.405531 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18cb8526-66f3-4822-9944-9d17828c4322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.422932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-18cb8526-66f3-4822-9944-9d17828c4322 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:09.424179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-18cb8526-66f3-4822-9944-9d17828c4322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:09.429261 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fc39f-95dc-4504-ad74-d37319889e74 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.429966 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fc39f-95dc-4504-ad74-d37319889e74 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.430236 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fc39f-95dc-4504-ad74-d37319889e74 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.430519 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1fc39f-95dc-4504-ad74-d37319889e74 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.431305 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1fc39f-95dc-4504-ad74-d37319889e74 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:09.433009 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1fc39f-95dc-4504-ad74-d37319889e74 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:09.443249 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1369/2731] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:16:09.447356 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fc39f-95dc-4504-ad74-d37319889e74 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.449231 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3b1fc39f-95dc-4504-ad74-d37319889e74 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:09.463193 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1fc39f-95dc-4504-ad74-d37319889e74 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.467840 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf27f897-ced9-4bb6-9c12-3ac5dbb0a8c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.468731 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27f897-ced9-4bb6-9c12-3ac5dbb0a8c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.469038 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27f897-ced9-4bb6-9c12-3ac5dbb0a8c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.469296 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf27f897-ced9-4bb6-9c12-3ac5dbb0a8c2 admin admin] PATH_INFO: `/v3/projects/6c418e32c7704f2684bd73292e698948/users/f4940d20aaf0493986e31a41db5eb1e0/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.483829 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27f897-ced9-4bb6-9c12-3ac5dbb0a8c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c418e32c7704f2684bd73292e698948, user_id=f4940d20aaf0493986e31a41db5eb1e0, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:09.484259 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1363/2732] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:16:09.486592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf27f897-ced9-4bb6-9c12-3ac5dbb0a8c2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:09.493010 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941902b-ede2-4d72-9fa3-faaec0f83df5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.493400 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941902b-ede2-4d72-9fa3-faaec0f83df5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.493701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3941902b-ede2-4d72-9fa3-faaec0f83df5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.502181 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1370/2733] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:09 2022] PUT /identity/v3/projects/6c418e32c7704f2684bd73292e698948/users/f4940d20aaf0493986e31a41db5eb1e0/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:16:09.513849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5ad17-ac26-4606-bd7a-6edce12f290f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.514128 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5ad17-ac26-4606-bd7a-6edce12f290f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.514411 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbf5ad17-ac26-4606-bd7a-6edce12f290f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.518400 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-3941902b-ede2-4d72-9fa3-faaec0f83df5 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:09.519664 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3941902b-ede2-4d72-9fa3-faaec0f83df5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:09.540183 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-dbf5ad17-ac26-4606-bd7a-6edce12f290f None None] MFA Rules not processed for user `f4940d20aaf0493986e31a41db5eb1e0`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:09.541366 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-dbf5ad17-ac26-4606-bd7a-6edce12f290f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:09.570812 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:09.582496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1364/2734] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:09.586678 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941902b-ede2-4d72-9fa3-faaec0f83df5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.595598 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3941902b-ede2-4d72-9fa3-faaec0f83df5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.600888 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb782a7-3ffd-41b4-8d63-027a865fc3df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.601839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb782a7-3ffd-41b4-8d63-027a865fc3df admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.602133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb782a7-3ffd-41b4-8d63-027a865fc3df admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.602418 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecb782a7-3ffd-41b4-8d63-027a865fc3df admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.603439 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb782a7-3ffd-41b4-8d63-027a865fc3df admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:16:09.603940 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb782a7-3ffd-41b4-8d63-027a865fc3df admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:09.605575 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecb782a7-3ffd-41b4-8d63-027a865fc3df admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:09.606244 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1371/2735] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:09 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:16:09.607270 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:09.611028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5ad17-ac26-4606-bd7a-6edce12f290f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.619181 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1365/2736] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:16:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:16:09.622697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbf5ad17-ac26-4606-bd7a-6edce12f290f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.623638 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb782a7-3ffd-41b4-8d63-027a865fc3df admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.627959 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936642dd-6109-4808-bed0-32ab813cec9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.628982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936642dd-6109-4808-bed0-32ab813cec9f admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.629267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936642dd-6109-4808-bed0-32ab813cec9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.629575 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-936642dd-6109-4808-bed0-32ab813cec9f admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.630275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936642dd-6109-4808-bed0-32ab813cec9f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:09.632270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-936642dd-6109-4808-bed0-32ab813cec9f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:09.634505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecb782a7-3ffd-41b4-8d63-027a865fc3df admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.639540 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ded39-8deb-4d75-b296-8d26f4000b0c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.640236 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ded39-8deb-4d75-b296-8d26f4000b0c admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.640533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ded39-8deb-4d75-b296-8d26f4000b0c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.640824 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52ded39-8deb-4d75-b296-8d26f4000b0c admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.641523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ded39-8deb-4d75-b296-8d26f4000b0c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:09.643493 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a52ded39-8deb-4d75-b296-8d26f4000b0c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:09.660943 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1372/2737] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:09 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:16:09.665520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936642dd-6109-4808-bed0-32ab813cec9f admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.674831 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1366/2738] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:09 2022] POST /identity/v3/projects => generated 389 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:16:09.675781 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ded39-8deb-4d75-b296-8d26f4000b0c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.727626 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-936642dd-6109-4808-bed0-32ab813cec9f admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.737928 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52ded39-8deb-4d75-b296-8d26f4000b0c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.744398 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ae0c3-be8b-4583-a112-6e79512d2e95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.745094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ae0c3-be8b-4583-a112-6e79512d2e95 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.745387 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ae0c3-be8b-4583-a112-6e79512d2e95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.745673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5ae0c3-be8b-4583-a112-6e79512d2e95 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.746382 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ae0c3-be8b-4583-a112-6e79512d2e95 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:09.748424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5ae0c3-be8b-4583-a112-6e79512d2e95 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:09.772851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1367/2739] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:09 2022] POST /identity/v3/users => generated 295 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:09.772851 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ae0c3-be8b-4583-a112-6e79512d2e95 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.773169 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3a5ae0c3-be8b-4583-a112-6e79512d2e95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:09.785258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5ae0c3-be8b-4583-a112-6e79512d2e95 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.789777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17762d-bd9b-4731-bc11-0429b0f9233a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.789986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a1e51-cada-4758-a732-38a377c60fed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.790655 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17762d-bd9b-4731-bc11-0429b0f9233a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.790769 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a1e51-cada-4758-a732-38a377c60fed admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.790953 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a1e51-cada-4758-a732-38a377c60fed admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.791056 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17762d-bd9b-4731-bc11-0429b0f9233a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.791235 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d17762d-bd9b-4731-bc11-0429b0f9233a admin admin] PATH_INFO: `/v3/users/576375cde39245cb89ab7f9ef8928270` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.791341 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1a1e51-cada-4758-a732-38a377c60fed admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.791977 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a1e51-cada-4758-a732-38a377c60fed admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:09.792441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17762d-bd9b-4731-bc11-0429b0f9233a admin admin] RBAC: Authorizing `identity:delete_user(user_id=576375cde39245cb89ab7f9ef8928270)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:09.793969 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1a1e51-cada-4758-a732-38a377c60fed admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:09.794401 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d17762d-bd9b-4731-bc11-0429b0f9233a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:09.818564 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1373/2740] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:09 2022] POST /identity/v3/users => generated 300 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:16:09.818564 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a1e51-cada-4758-a732-38a377c60fed admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.831394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1a1e51-cada-4758-a732-38a377c60fed admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.836161 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a5d8c-99bd-4b31-89f6-1bec6ee1e953 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.836834 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a5d8c-99bd-4b31-89f6-1bec6ee1e953 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.837119 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a5d8c-99bd-4b31-89f6-1bec6ee1e953 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.837413 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb2a5d8c-99bd-4b31-89f6-1bec6ee1e953 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.838138 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a5d8c-99bd-4b31-89f6-1bec6ee1e953 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:09.839757 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb2a5d8c-99bd-4b31-89f6-1bec6ee1e953 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:09.844451 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1d17762d-bd9b-4731-bc11-0429b0f9233a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '576375cde39245cb89ab7f9ef8928270'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:09.849970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1374/2741] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:09.852969 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a5d8c-99bd-4b31-89f6-1bec6ee1e953 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.856942 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1d17762d-bd9b-4731-bc11-0429b0f9233a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '576375cde39245cb89ab7f9ef8928270'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:09.862613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb2a5d8c-99bd-4b31-89f6-1bec6ee1e953 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.865889 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1d17762d-bd9b-4731-bc11-0429b0f9233a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '576375cde39245cb89ab7f9ef8928270'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:09.866570 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1d17762d-bd9b-4731-bc11-0429b0f9233a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '576375cde39245cb89ab7f9ef8928270'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:09.867417 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ad7295-eee6-4bfe-b03e-a5cf31011bd8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.868096 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad7295-eee6-4bfe-b03e-a5cf31011bd8 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.868387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad7295-eee6-4bfe-b03e-a5cf31011bd8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.868673 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ad7295-eee6-4bfe-b03e-a5cf31011bd8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.869413 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad7295-eee6-4bfe-b03e-a5cf31011bd8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:09.871056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ad7295-eee6-4bfe-b03e-a5cf31011bd8 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:09.874449 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1d17762d-bd9b-4731-bc11-0429b0f9233a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '576375cde39245cb89ab7f9ef8928270'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:09.880910 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1368/2742] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:16:09 2022] DELETE /identity/v3/users/576375cde39245cb89ab7f9ef8928270 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:09.883874 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17762d-bd9b-4731-bc11-0429b0f9233a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.884408 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1375/2743] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:09.884521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1d17762d-bd9b-4731-bc11-0429b0f9233a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:09.885169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96a772-e284-4ad4-9f4c-52b3993c5de3 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.885462 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96a772-e284-4ad4-9f4c-52b3993c5de3 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.885739 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df96a772-e284-4ad4-9f4c-52b3993c5de3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.902342 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-df96a772-e284-4ad4-9f4c-52b3993c5de3 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:09.903528 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-df96a772-e284-4ad4-9f4c-52b3993c5de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:09.919097 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=85,keystone:INSERT=5,keystone:DELETE=10 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:09.922745 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:09.950042 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d17762d-bd9b-4731-bc11-0429b0f9233a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.954744 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d61a3-e9f9-4ce5-8e90-8c89a44d66c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.955430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d61a3-e9f9-4ce5-8e90-8c89a44d66c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.955723 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d61a3-e9f9-4ce5-8e90-8c89a44d66c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.956033 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f3d61a3-e9f9-4ce5-8e90-8c89a44d66c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.956769 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d61a3-e9f9-4ce5-8e90-8c89a44d66c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:09.958396 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f3d61a3-e9f9-4ce5-8e90-8c89a44d66c7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:09.965409 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1376/2744] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:16:09.966259 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96a772-e284-4ad4-9f4c-52b3993c5de3 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.967608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-df96a772-e284-4ad4-9f4c-52b3993c5de3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:09.970908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1369/2745] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:09 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:09.975121 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d61a3-e9f9-4ce5-8e90-8c89a44d66c7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:09.983218 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df96a772-e284-4ad4-9f4c-52b3993c5de3 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.985096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f3d61a3-e9f9-4ce5-8e90-8c89a44d66c7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:09.988402 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8c13c-8d20-4cd0-afcb-8669492f3422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.989313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8c13c-8d20-4cd0-afcb-8669492f3422 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.989601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8c13c-8d20-4cd0-afcb-8669492f3422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.989705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0c676-6921-474e-8795-5bd65174a39a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:09.989882 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def8c13c-8d20-4cd0-afcb-8669492f3422 admin admin] PATH_INFO: `/v3/projects/08ebf84cf7b14027bcd08ddb4aff5651/users/0fa8dcb5119949b68eb7c91fa552bf70/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:09.990548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0c676-6921-474e-8795-5bd65174a39a admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:09.990835 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0c676-6921-474e-8795-5bd65174a39a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:09.991127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e0c676-6921-474e-8795-5bd65174a39a admin admin] PATH_INFO: `/v3/projects/852905969b234e5386566425166c1f7f/users/4e864a9b49f24af9b7f517ddddd01776/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:10.005905 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8c13c-8d20-4cd0-afcb-8669492f3422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ebf84cf7b14027bcd08ddb4aff5651, user_id=0fa8dcb5119949b68eb7c91fa552bf70, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:10.006446 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0c676-6921-474e-8795-5bd65174a39a admin admin] RBAC: Authorizing `identity:create_grant(project_id=852905969b234e5386566425166c1f7f, user_id=4e864a9b49f24af9b7f517ddddd01776, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:10.008611 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-def8c13c-8d20-4cd0-afcb-8669492f3422 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:10.009158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54e0c676-6921-474e-8795-5bd65174a39a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:10.027224 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1377/2746] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:09 2022] PUT /identity/v3/projects/08ebf84cf7b14027bcd08ddb4aff5651/users/0fa8dcb5119949b68eb7c91fa552bf70/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:10.027405 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1370/2747] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:09 2022] PUT /identity/v3/projects/852905969b234e5386566425166c1f7f/users/4e864a9b49f24af9b7f517ddddd01776/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:16:10.027495 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8c13c-8d20-4cd0-afcb-8669492f3422 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:10.027495 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-def8c13c-8d20-4cd0-afcb-8669492f3422 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:10.029066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0c676-6921-474e-8795-5bd65174a39a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:10.086802 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def8c13c-8d20-4cd0-afcb-8669492f3422 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:10.096096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e0c676-6921-474e-8795-5bd65174a39a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:10.100892 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-978895dd-cf23-4e6d-a95d-5ec51db04b7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:10.101575 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978895dd-cf23-4e6d-a95d-5ec51db04b7d admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:10.101872 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978895dd-cf23-4e6d-a95d-5ec51db04b7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:10.102158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-978895dd-cf23-4e6d-a95d-5ec51db04b7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:10.102899 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978895dd-cf23-4e6d-a95d-5ec51db04b7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:10.104643 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-978895dd-cf23-4e6d-a95d-5ec51db04b7d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:10.114881 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1371/2748] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:10 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:16:10.116137 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488be6b0-e75d-4970-a4d3-f9e2028f7347 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:10.116436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488be6b0-e75d-4970-a4d3-f9e2028f7347 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:10.116724 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-488be6b0-e75d-4970-a4d3-f9e2028f7347 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:10.139552 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac59de-b448-42dc-ba9f-076afc7ea412 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:10.140250 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac59de-b448-42dc-ba9f-076afc7ea412 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:10.140542 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac59de-b448-42dc-ba9f-076afc7ea412 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:10.140822 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ac59de-b448-42dc-ba9f-076afc7ea412 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:10.143355 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-488be6b0-e75d-4970-a4d3-f9e2028f7347 None None] MFA Rules not processed for user `4e864a9b49f24af9b7f517ddddd01776`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:10.144619 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-488be6b0-e75d-4970-a4d3-f9e2028f7347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:10.149932 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ac59de-b448-42dc-ba9f-076afc7ea412 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:10.151376 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ac59de-b448-42dc-ba9f-076afc7ea412 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:10.160881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1378/2749] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:10.161630 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac59de-b448-42dc-ba9f-076afc7ea412 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:10.170568 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ac59de-b448-42dc-ba9f-076afc7ea412 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:10.175404 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80cbbf44-682e-4b92-866e-895c6732082f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:10.176330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbbf44-682e-4b92-866e-895c6732082f admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:10.176619 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbbf44-682e-4b92-866e-895c6732082f admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:10.176903 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80cbbf44-682e-4b92-866e-895c6732082f admin admin] PATH_INFO: `/v3/projects/08ebf84cf7b14027bcd08ddb4aff5651/users/0fa8dcb5119949b68eb7c91fa552bf70/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:10.179338 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbbf44-682e-4b92-866e-895c6732082f admin admin] RBAC: Authorizing `identity:create_grant(project_id=08ebf84cf7b14027bcd08ddb4aff5651, user_id=0fa8dcb5119949b68eb7c91fa552bf70, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:10.182205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80cbbf44-682e-4b92-866e-895c6732082f admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:10.193970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1379/2750] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:10 2022] PUT /identity/v3/projects/08ebf84cf7b14027bcd08ddb4aff5651/users/0fa8dcb5119949b68eb7c91fa552bf70/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:10.205143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc96dc4-d0ac-4884-b3a8-cf32d0db8c13 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:10.205585 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc96dc4-d0ac-4884-b3a8-cf32d0db8c13 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:10.206260 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbc96dc4-d0ac-4884-b3a8-cf32d0db8c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:10.218383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1372/2751] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:10 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:10.231850 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488be6b0-e75d-4970-a4d3-f9e2028f7347 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:10.233453 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-488be6b0-e75d-4970-a4d3-f9e2028f7347 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:10.238844 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-dbc96dc4-d0ac-4884-b3a8-cf32d0db8c13 None None] MFA Rules not processed for user `0fa8dcb5119949b68eb7c91fa552bf70`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:10.240718 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-dbc96dc4-d0ac-4884-b3a8-cf32d0db8c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:10.277777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:10.287967 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:10.315451 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-488be6b0-e75d-4970-a4d3-f9e2028f7347 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:10.337633 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1380/2752] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:10 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:16:10.350946 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc96dc4-d0ac-4884-b3a8-cf32d0db8c13 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:10.353336 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-dbc96dc4-d0ac-4884-b3a8-cf32d0db8c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:10.385767 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70b3df-17ff-4596-b86e-9f96e94a855f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:10.386490 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70b3df-17ff-4596-b86e-9f96e94a855f service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:10.386783 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70b3df-17ff-4596-b86e-9f96e94a855f service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:10.387075 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a70b3df-17ff-4596-b86e-9f96e94a855f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:10.415467 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbc96dc4-d0ac-4884-b3a8-cf32d0db8c13 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:10.419534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011dea14-4232-4768-b50e-e4c43aee1d9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:10.420292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dea14-4232-4768-b50e-e4c43aee1d9d service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:10.420588 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dea14-4232-4768-b50e-e4c43aee1d9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:10.420874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-011dea14-4232-4768-b50e-e4c43aee1d9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:10.431163 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011dea14-4232-4768-b50e-e4c43aee1d9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:10.432534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-011dea14-4232-4768-b50e-e4c43aee1d9d service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:10.444252 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1381/2753] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:10 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:16:10.452211 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70b3df-17ff-4596-b86e-9f96e94a855f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:10.453698 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a70b3df-17ff-4596-b86e-9f96e94a855f service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:10.465946 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1373/2754] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:10 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 235 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:16:10.481019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011dea14-4232-4768-b50e-e4c43aee1d9d service cinder] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:10.482420 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-011dea14-4232-4768-b50e-e4c43aee1d9d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:10.548130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70b3df-17ff-4596-b86e-9f96e94a855f service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:10.574183 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-011dea14-4232-4768-b50e-e4c43aee1d9d service cinder] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:10.579103 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:10.580449 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:10.580614 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:10.580906 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328 admin admin] PATH_INFO: `/v3/projects/7b3ab2fac8024c19abc57298984fd12a` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:10.582152 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7b3ab2fac8024c19abc57298984fd12a)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:10.584121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:10.629784 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a70b3df-17ff-4596-b86e-9f96e94a855f service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:10.637479 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7b3ab2fac8024c19abc57298984fd12a', 'request_id': 'req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:10.644102 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7b3ab2fac8024c19abc57298984fd12a', 'request_id': 'req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:10.653195 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7b3ab2fac8024c19abc57298984fd12a', 'request_id': 'req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:10.666866 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328 admin admin] The token cache is being invalidate because project 7b3ab2fac8024c19abc57298984fd12a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:10.667182 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:10.667863 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:10.669744 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1382/2755] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:10 2022] DELETE /identity/v3/projects/7b3ab2fac8024c19abc57298984fd12a => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:10.673553 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:10.675014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:10.681228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:10.695362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:10.709926 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b9ae68-b0b9-48f9-bb9e-ac5d7e1ed40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:10.710906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9ae68-b0b9-48f9-bb9e-ac5d7e1ed40f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:10.711403 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9ae68-b0b9-48f9-bb9e-ac5d7e1ed40f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:10.711764 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b9ae68-b0b9-48f9-bb9e-ac5d7e1ed40f admin admin] PATH_INFO: `/v3/users/f4940d20aaf0493986e31a41db5eb1e0` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:10.729015 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9ae68-b0b9-48f9-bb9e-ac5d7e1ed40f admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4940d20aaf0493986e31a41db5eb1e0)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:10.731268 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b9ae68-b0b9-48f9-bb9e-ac5d7e1ed40f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:10.756229 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b437b6e-0f06-4241-a1fd-e1d96c7b4328 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:10.764914 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cb632-de38-4d91-ab4c-092c698347df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:10.766373 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cb632-de38-4d91-ab4c-092c698347df admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:10.766495 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cb632-de38-4d91-ab4c-092c698347df admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:10.766818 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-100cb632-de38-4d91-ab4c-092c698347df admin admin] PATH_INFO: `/v3/users/da23817503854d668de4ac813ad99473` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:10.768350 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100cb632-de38-4d91-ab4c-092c698347df admin admin] RBAC: Authorizing `identity:delete_user(user_id=da23817503854d668de4ac813ad99473)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:10.772671 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-100cb632-de38-4d91-ab4c-092c698347df admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:10.793076 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-92b9ae68-b0b9-48f9-bb9e-ac5d7e1ed40f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4940d20aaf0493986e31a41db5eb1e0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:10.816657 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-92b9ae68-b0b9-48f9-bb9e-ac5d7e1ed40f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4940d20aaf0493986e31a41db5eb1e0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:10.826016 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-92b9ae68-b0b9-48f9-bb9e-ac5d7e1ed40f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4940d20aaf0493986e31a41db5eb1e0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:10.826702 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-92b9ae68-b0b9-48f9-bb9e-ac5d7e1ed40f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4940d20aaf0493986e31a41db5eb1e0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:10.834454 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-92b9ae68-b0b9-48f9-bb9e-ac5d7e1ed40f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4940d20aaf0493986e31a41db5eb1e0'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:10.837210 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-100cb632-de38-4d91-ab4c-092c698347df admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'da23817503854d668de4ac813ad99473'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:10.837838 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-100cb632-de38-4d91-ab4c-092c698347df admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'da23817503854d668de4ac813ad99473'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:10.841827 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1374/2756] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:16:10 2022] DELETE /identity/v3/users/f4940d20aaf0493986e31a41db5eb1e0 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:16:10.846739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbe7df-d6cb-43b9-8e8a-e65e2e443e28 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:10.847105 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbe7df-d6cb-43b9-8e8a-e65e2e443e28 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:10.847449 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbe7df-d6cb-43b9-8e8a-e65e2e443e28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:10.855317 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-100cb632-de38-4d91-ab4c-092c698347df admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'da23817503854d668de4ac813ad99473'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:10.866432 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-100cb632-de38-4d91-ab4c-092c698347df admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'da23817503854d668de4ac813ad99473'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:10.867287 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-12fbe7df-d6cb-43b9-8e8a-e65e2e443e28 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:10.867442 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-100cb632-de38-4d91-ab4c-092c698347df admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'da23817503854d668de4ac813ad99473'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:10.868564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-12fbe7df-d6cb-43b9-8e8a-e65e2e443e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:10.882185 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1383/2757] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:16:10 2022] DELETE /identity/v3/users/da23817503854d668de4ac813ad99473 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:10.930309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1375/2758] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:16:10.941348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cb632-de38-4d91-ab4c-092c698347df admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:10.943058 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-100cb632-de38-4d91-ab4c-092c698347df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:11.010388 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-100cb632-de38-4d91-ab4c-092c698347df admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:11.014390 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cc59f2-8e81-48eb-a218-92d1e201f777 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:11.015068 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc59f2-8e81-48eb-a218-92d1e201f777 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:11.015351 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc59f2-8e81-48eb-a218-92d1e201f777 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:11.015630 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cc59f2-8e81-48eb-a218-92d1e201f777 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:11.057930 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:11.084266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc59f2-8e81-48eb-a218-92d1e201f777 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:11.085936 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21cc59f2-8e81-48eb-a218-92d1e201f777 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:11.095884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1384/2759] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:11.207522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fbe7df-d6cb-43b9-8e8a-e65e2e443e28 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:11.209031 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-12fbe7df-d6cb-43b9-8e8a-e65e2e443e28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:11.224840 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fbe7df-d6cb-43b9-8e8a-e65e2e443e28 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:11.236224 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b230d8-1c05-4f44-afd4-419fb752e14c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:11.237373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b230d8-1c05-4f44-afd4-419fb752e14c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:11.237663 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b230d8-1c05-4f44-afd4-419fb752e14c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:11.237962 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72b230d8-1c05-4f44-afd4-419fb752e14c admin admin] PATH_INFO: `/v3/projects/5c10363f62a04717a7a3f9c3f1862ed6` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:11.239251 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b230d8-1c05-4f44-afd4-419fb752e14c admin admin] RBAC: Authorizing `identity:delete_project(project_id=5c10363f62a04717a7a3f9c3f1862ed6)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:11.241339 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72b230d8-1c05-4f44-afd4-419fb752e14c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:11.306101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-72b230d8-1c05-4f44-afd4-419fb752e14c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c10363f62a04717a7a3f9c3f1862ed6', 'request_id': 'req-72b230d8-1c05-4f44-afd4-419fb752e14c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:11.315938 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-72b230d8-1c05-4f44-afd4-419fb752e14c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c10363f62a04717a7a3f9c3f1862ed6', 'request_id': 'req-72b230d8-1c05-4f44-afd4-419fb752e14c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:11.322999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-72b230d8-1c05-4f44-afd4-419fb752e14c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c10363f62a04717a7a3f9c3f1862ed6', 'request_id': 'req-72b230d8-1c05-4f44-afd4-419fb752e14c'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:11.325156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:11.335269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-72b230d8-1c05-4f44-afd4-419fb752e14c admin admin] The token cache is being invalidate because project 5c10363f62a04717a7a3f9c3f1862ed6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:11.335566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-72b230d8-1c05-4f44-afd4-419fb752e14c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:11.337209 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-72b230d8-1c05-4f44-afd4-419fb752e14c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:11.339086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1376/2760] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:11 2022] DELETE /identity/v3/projects/5c10363f62a04717a7a3f9c3f1862ed6 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:11.344311 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52719d-bd39-4152-aa90-b7b96ef1bcbc None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:11.344606 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52719d-bd39-4152-aa90-b7b96ef1bcbc None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:11.344896 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b52719d-bd39-4152-aa90-b7b96ef1bcbc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:11.363394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-6b52719d-bd39-4152-aa90-b7b96ef1bcbc None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:11.364641 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-6b52719d-bd39-4152-aa90-b7b96ef1bcbc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:11.419561 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b230d8-1c05-4f44-afd4-419fb752e14c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:11.421241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-72b230d8-1c05-4f44-afd4-419fb752e14c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:11.431018 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1385/2761] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:11.439450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb25757-5ee8-42fc-a0af-d318b90b9d5f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:11.439850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb25757-5ee8-42fc-a0af-d318b90b9d5f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:11.440374 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eb25757-5ee8-42fc-a0af-d318b90b9d5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:11.443794 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72b230d8-1c05-4f44-afd4-419fb752e14c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:11.448581 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0a994-6a62-442b-b3cb-7fcd241986b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:11.449464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0a994-6a62-442b-b3cb-7fcd241986b6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:11.449778 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0a994-6a62-442b-b3cb-7fcd241986b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:11.450018 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f0a994-6a62-442b-b3cb-7fcd241986b6 admin admin] PATH_INFO: `/v3/projects/6c418e32c7704f2684bd73292e698948` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:11.451343 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0a994-6a62-442b-b3cb-7fcd241986b6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c418e32c7704f2684bd73292e698948)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:11.453381 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f0a994-6a62-442b-b3cb-7fcd241986b6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:11.463417 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-4eb25757-5ee8-42fc-a0af-d318b90b9d5f None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:11.464687 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4eb25757-5ee8-42fc-a0af-d318b90b9d5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:11.472690 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1386/2762] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:16:11.476787 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb25757-5ee8-42fc-a0af-d318b90b9d5f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:11.485824 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eb25757-5ee8-42fc-a0af-d318b90b9d5f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:11.490645 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b878e3-7401-4d8a-9ffc-98f3f3b0c202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:11.491304 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b878e3-7401-4d8a-9ffc-98f3f3b0c202 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:11.491587 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b878e3-7401-4d8a-9ffc-98f3f3b0c202 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:11.491893 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b878e3-7401-4d8a-9ffc-98f3f3b0c202 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:11.492628 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b878e3-7401-4d8a-9ffc-98f3f3b0c202 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:16:11.492939 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b878e3-7401-4d8a-9ffc-98f3f3b0c202 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:11.494541 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b878e3-7401-4d8a-9ffc-98f3f3b0c202 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:11.502242 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-60f0a994-6a62-442b-b3cb-7fcd241986b6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c418e32c7704f2684bd73292e698948', 'request_id': 'req-60f0a994-6a62-442b-b3cb-7fcd241986b6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:11.506272 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1387/2763] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:16:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:16:11.510519 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b878e3-7401-4d8a-9ffc-98f3f3b0c202 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:11.512190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-60f0a994-6a62-442b-b3cb-7fcd241986b6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c418e32c7704f2684bd73292e698948', 'request_id': 'req-60f0a994-6a62-442b-b3cb-7fcd241986b6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:11.518981 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-60f0a994-6a62-442b-b3cb-7fcd241986b6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c418e32c7704f2684bd73292e698948', 'request_id': 'req-60f0a994-6a62-442b-b3cb-7fcd241986b6'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:11.520128 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b878e3-7401-4d8a-9ffc-98f3f3b0c202 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:11.530505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-60f0a994-6a62-442b-b3cb-7fcd241986b6 admin admin] The token cache is being invalidate because project 6c418e32c7704f2684bd73292e698948 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:11.530910 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-60f0a994-6a62-442b-b3cb-7fcd241986b6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:11.531731 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-60f0a994-6a62-442b-b3cb-7fcd241986b6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:11.533672 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1377/2764] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:11 2022] DELETE /identity/v3/projects/6c418e32c7704f2684bd73292e698948 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:16:11.537628 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0a994-6a62-442b-b3cb-7fcd241986b6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:11.539001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-60f0a994-6a62-442b-b3cb-7fcd241986b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:11.582482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3903d9-66e8-4650-87dc-f160a31a95c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:11.583164 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3903d9-66e8-4650-87dc-f160a31a95c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:11.583456 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3903d9-66e8-4650-87dc-f160a31a95c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:11.583737 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3903d9-66e8-4650-87dc-f160a31a95c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:11.584471 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3903d9-66e8-4650-87dc-f160a31a95c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:11.586399 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3903d9-66e8-4650-87dc-f160a31a95c1 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:11.608817 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f0a994-6a62-442b-b3cb-7fcd241986b6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:11.617074 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1388/2765] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:11 2022] POST /identity/v3/projects => generated 375 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:16:11.617253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048490c1-5302-4857-a723-cd20dd91a33e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:11.617253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048490c1-5302-4857-a723-cd20dd91a33e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:11.617253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048490c1-5302-4857-a723-cd20dd91a33e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:11.617253 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048490c1-5302-4857-a723-cd20dd91a33e admin admin] PATH_INFO: `/v3/users/0fa8dcb5119949b68eb7c91fa552bf70` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:11.617619 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048490c1-5302-4857-a723-cd20dd91a33e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fa8dcb5119949b68eb7c91fa552bf70)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:11.618173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3903d9-66e8-4650-87dc-f160a31a95c1 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:11.618421 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048490c1-5302-4857-a723-cd20dd91a33e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:11.619867 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-cf3903d9-66e8-4650-87dc-f160a31a95c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:11.662912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:11.671937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-048490c1-5302-4857-a723-cd20dd91a33e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fa8dcb5119949b68eb7c91fa552bf70'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:11.684815 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-048490c1-5302-4857-a723-cd20dd91a33e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fa8dcb5119949b68eb7c91fa552bf70'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:11.686904 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3903d9-66e8-4650-87dc-f160a31a95c1 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:11.692621 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41cfd4-63ef-4cbd-b3a1-849236b412d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:11.693277 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41cfd4-63ef-4cbd-b3a1-849236b412d6 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:11.693419 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-048490c1-5302-4857-a723-cd20dd91a33e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fa8dcb5119949b68eb7c91fa552bf70'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:11.693569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41cfd4-63ef-4cbd-b3a1-849236b412d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:11.693849 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd41cfd4-63ef-4cbd-b3a1-849236b412d6 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:11.694059 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-048490c1-5302-4857-a723-cd20dd91a33e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fa8dcb5119949b68eb7c91fa552bf70'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:11.694592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41cfd4-63ef-4cbd-b3a1-849236b412d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:11.696933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd41cfd4-63ef-4cbd-b3a1-849236b412d6 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:11.698904 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:11.702474 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-048490c1-5302-4857-a723-cd20dd91a33e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fa8dcb5119949b68eb7c91fa552bf70'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:11.709868 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1378/2766] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:16:11 2022] DELETE /identity/v3/users/0fa8dcb5119949b68eb7c91fa552bf70 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:16:11.724246 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1389/2767] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:11 2022] POST /identity/v3/users => generated 288 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:16:11.725609 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048490c1-5302-4857-a723-cd20dd91a33e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:11.727440 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-048490c1-5302-4857-a723-cd20dd91a33e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:11.794373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048490c1-5302-4857-a723-cd20dd91a33e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:11.800032 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21ce90-9b26-4e64-8e58-5a8942fc9cd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:11.800730 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21ce90-9b26-4e64-8e58-5a8942fc9cd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:11.801022 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21ce90-9b26-4e64-8e58-5a8942fc9cd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:11.801310 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f21ce90-9b26-4e64-8e58-5a8942fc9cd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:11.802061 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21ce90-9b26-4e64-8e58-5a8942fc9cd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:11.803744 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f21ce90-9b26-4e64-8e58-5a8942fc9cd7 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:11.815498 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1379/2768] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:16:11.819759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41cfd4-63ef-4cbd-b3a1-849236b412d6 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:11.832377 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd41cfd4-63ef-4cbd-b3a1-849236b412d6 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:11.837894 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1892a7ed-2872-4749-8c26-1f8a7abaee91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:11.838828 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a7ed-2872-4749-8c26-1f8a7abaee91 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:11.839124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a7ed-2872-4749-8c26-1f8a7abaee91 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:11.839442 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1892a7ed-2872-4749-8c26-1f8a7abaee91 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:11.840817 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892a7ed-2872-4749-8c26-1f8a7abaee91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:11.842750 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1892a7ed-2872-4749-8c26-1f8a7abaee91 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:11.867193 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1390/2769] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:11 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:16:11.873310 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21ce90-9b26-4e64-8e58-5a8942fc9cd7 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:11.884505 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f21ce90-9b26-4e64-8e58-5a8942fc9cd7 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:11.889891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309fc844-f598-4969-a830-73b623f688be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:11.890904 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fc844-f598-4969-a830-73b623f688be admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:11.891740 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fc844-f598-4969-a830-73b623f688be admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:11.891740 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309fc844-f598-4969-a830-73b623f688be admin admin] PATH_INFO: `/v3/projects/da13efd30db34a93a36b81ff8154923d/users/daa5a4286c5e421498bbf8d1c7cdd6b7/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:11.908985 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309fc844-f598-4969-a830-73b623f688be admin admin] RBAC: Authorizing `identity:create_grant(project_id=da13efd30db34a93a36b81ff8154923d, user_id=daa5a4286c5e421498bbf8d1c7cdd6b7, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:11.911802 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309fc844-f598-4969-a830-73b623f688be admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:11.928430 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1380/2770] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:11 2022] PUT /identity/v3/projects/da13efd30db34a93a36b81ff8154923d/users/daa5a4286c5e421498bbf8d1c7cdd6b7/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:11.940632 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f711f-a235-4e37-ae70-5826397398a2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:11.941034 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f711f-a235-4e37-ae70-5826397398a2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:11.941334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8f711f-a235-4e37-ae70-5826397398a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:11.972656 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-fc8f711f-a235-4e37-ae70-5826397398a2 None None] MFA Rules not processed for user `daa5a4286c5e421498bbf8d1c7cdd6b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:11.973865 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-fc8f711f-a235-4e37-ae70-5826397398a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:11.990391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309fc844-f598-4969-a830-73b623f688be admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:11.992836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-309fc844-f598-4969-a830-73b623f688be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:12.070939 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1391/2771] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:12.078823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f711f-a235-4e37-ae70-5826397398a2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:12.080933 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-fc8f711f-a235-4e37-ae70-5826397398a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:12.103176 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309fc844-f598-4969-a830-73b623f688be admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:12.108610 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a28ca70-7cdb-4f6a-8ffb-dbc639ba23b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:12.109491 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28ca70-7cdb-4f6a-8ffb-dbc639ba23b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:12.109784 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28ca70-7cdb-4f6a-8ffb-dbc639ba23b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:12.110071 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a28ca70-7cdb-4f6a-8ffb-dbc639ba23b4 admin admin] PATH_INFO: `/v3/projects/08ebf84cf7b14027bcd08ddb4aff5651` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:12.111328 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28ca70-7cdb-4f6a-8ffb-dbc639ba23b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=08ebf84cf7b14027bcd08ddb4aff5651)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:12.113342 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a28ca70-7cdb-4f6a-8ffb-dbc639ba23b4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:12.139268 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc8f711f-a235-4e37-ae70-5826397398a2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:12.143294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fb6328-d91c-41f4-9eb3-7f6bde764999 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:12.143985 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb6328-d91c-41f4-9eb3-7f6bde764999 service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:12.144282 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb6328-d91c-41f4-9eb3-7f6bde764999 service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:12.144600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fb6328-d91c-41f4-9eb3-7f6bde764999 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:12.153651 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb6328-d91c-41f4-9eb3-7f6bde764999 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:12.155157 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fb6328-d91c-41f4-9eb3-7f6bde764999 service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:12.161116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3a28ca70-7cdb-4f6a-8ffb-dbc639ba23b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '08ebf84cf7b14027bcd08ddb4aff5651', 'request_id': 'req-3a28ca70-7cdb-4f6a-8ffb-dbc639ba23b4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:12.165298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1392/2772] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) May 25 16:16:12.169224 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3a28ca70-7cdb-4f6a-8ffb-dbc639ba23b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '08ebf84cf7b14027bcd08ddb4aff5651', 'request_id': 'req-3a28ca70-7cdb-4f6a-8ffb-dbc639ba23b4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:12.176341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3a28ca70-7cdb-4f6a-8ffb-dbc639ba23b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '08ebf84cf7b14027bcd08ddb4aff5651', 'request_id': 'req-3a28ca70-7cdb-4f6a-8ffb-dbc639ba23b4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:12.182127 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:12.190139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3a28ca70-7cdb-4f6a-8ffb-dbc639ba23b4 admin admin] The token cache is being invalidate because project 08ebf84cf7b14027bcd08ddb4aff5651 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:12.190472 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3a28ca70-7cdb-4f6a-8ffb-dbc639ba23b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:12.191927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-3a28ca70-7cdb-4f6a-8ffb-dbc639ba23b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:12.193768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1381/2773] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:11 2022] DELETE /identity/v3/projects/08ebf84cf7b14027bcd08ddb4aff5651 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:12.198728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841a6c2-fd93-4f96-86a9-b4acc975e546 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:12.199021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841a6c2-fd93-4f96-86a9-b4acc975e546 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:12.199300 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c841a6c2-fd93-4f96-86a9-b4acc975e546 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:12.218169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c841a6c2-fd93-4f96-86a9-b4acc975e546 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:12.219450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c841a6c2-fd93-4f96-86a9-b4acc975e546 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:12.265191 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:12.281177 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1393/2774] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:16:12.290551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a4110-9c00-4f3f-aabe-86afdc706d89 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:12.290857 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a4110-9c00-4f3f-aabe-86afdc706d89 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:12.291225 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a4110-9c00-4f3f-aabe-86afdc706d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:12.316133 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-898a4110-9c00-4f3f-aabe-86afdc706d89 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:12.317548 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-898a4110-9c00-4f3f-aabe-86afdc706d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:12.326849 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1382/2775] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:12.331365 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841a6c2-fd93-4f96-86a9-b4acc975e546 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:12.341208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c841a6c2-fd93-4f96-86a9-b4acc975e546 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:12.345874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03d65c-8752-4e80-ab7f-1f09edf3340a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:12.346596 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03d65c-8752-4e80-ab7f-1f09edf3340a admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:12.346890 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03d65c-8752-4e80-ab7f-1f09edf3340a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:12.347173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03d65c-8752-4e80-ab7f-1f09edf3340a admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:12.348015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03d65c-8752-4e80-ab7f-1f09edf3340a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:16:12.348330 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03d65c-8752-4e80-ab7f-1f09edf3340a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:12.350122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03d65c-8752-4e80-ab7f-1f09edf3340a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:12.363022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1394/2776] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:16:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) May 25 16:16:12.367268 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a4110-9c00-4f3f-aabe-86afdc706d89 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:12.378707 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a4110-9c00-4f3f-aabe-86afdc706d89 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:12.384733 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b062f0-e919-4988-b6fa-6826eb91541d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:12.385408 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b062f0-e919-4988-b6fa-6826eb91541d admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:12.385709 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b062f0-e919-4988-b6fa-6826eb91541d admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:12.385995 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b062f0-e919-4988-b6fa-6826eb91541d admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:12.386786 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b062f0-e919-4988-b6fa-6826eb91541d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:12.389156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b062f0-e919-4988-b6fa-6826eb91541d admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:12.423605 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1383/2777] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:12 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:12.428551 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03d65c-8752-4e80-ab7f-1f09edf3340a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:12.444806 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03d65c-8752-4e80-ab7f-1f09edf3340a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:12.526294 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec481f-f658-4247-823e-9c4289de72db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:12.526939 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec481f-f658-4247-823e-9c4289de72db admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:12.527232 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec481f-f658-4247-823e-9c4289de72db admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:12.527519 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cec481f-f658-4247-823e-9c4289de72db admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:12.528254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cec481f-f658-4247-823e-9c4289de72db admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:12.530228 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cec481f-f658-4247-823e-9c4289de72db admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:12.552303 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1395/2778] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:12 2022] POST /identity/v3/users => generated 293 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:16:12.556739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b062f0-e919-4988-b6fa-6826eb91541d admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:12.567424 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b062f0-e919-4988-b6fa-6826eb91541d admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:12.573110 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f276d-1abd-4b17-9a2f-01bdb505d3e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:12.573789 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f276d-1abd-4b17-9a2f-01bdb505d3e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:12.574076 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f276d-1abd-4b17-9a2f-01bdb505d3e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:12.574362 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170f276d-1abd-4b17-9a2f-01bdb505d3e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:12.575116 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f276d-1abd-4b17-9a2f-01bdb505d3e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:12.576836 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170f276d-1abd-4b17-9a2f-01bdb505d3e2 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:12.587519 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1384/2779] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:12.591609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec481f-f658-4247-823e-9c4289de72db admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:12.601648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cec481f-f658-4247-823e-9c4289de72db admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:12.606946 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81873796-8e2f-4ff0-8503-9d19b6d54ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:12.607623 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81873796-8e2f-4ff0-8503-9d19b6d54ce4 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:12.607941 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81873796-8e2f-4ff0-8503-9d19b6d54ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:12.608241 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81873796-8e2f-4ff0-8503-9d19b6d54ce4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:12.609114 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81873796-8e2f-4ff0-8503-9d19b6d54ce4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:12.610744 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81873796-8e2f-4ff0-8503-9d19b6d54ce4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:12.621067 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1396/2780] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:12 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:16:12.625678 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f276d-1abd-4b17-9a2f-01bdb505d3e2 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:12.636334 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170f276d-1abd-4b17-9a2f-01bdb505d3e2 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:12.642124 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9d04c-46ce-4ea1-9de4-da82659afcb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:12.643017 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9d04c-46ce-4ea1-9de4-da82659afcb0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:12.643305 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9d04c-46ce-4ea1-9de4-da82659afcb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:12.643584 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68a9d04c-46ce-4ea1-9de4-da82659afcb0 admin admin] PATH_INFO: `/v3/projects/fa382782425d44f9a654480c1a2794c3/users/ca84f75d8aa74efdb1e7d791cf5b9488/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:12.660169 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9d04c-46ce-4ea1-9de4-da82659afcb0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa382782425d44f9a654480c1a2794c3, user_id=ca84f75d8aa74efdb1e7d791cf5b9488, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:12.662857 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68a9d04c-46ce-4ea1-9de4-da82659afcb0 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:12.678307 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1385/2781] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:12 2022] PUT /identity/v3/projects/fa382782425d44f9a654480c1a2794c3/users/ca84f75d8aa74efdb1e7d791cf5b9488/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:12.688717 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbef572-2a5d-4a8f-bc17-fd835aae421e None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:12.689039 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbef572-2a5d-4a8f-bc17-fd835aae421e None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:12.689328 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdbef572-2a5d-4a8f-bc17-fd835aae421e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:12.716844 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-cdbef572-2a5d-4a8f-bc17-fd835aae421e None None] MFA Rules not processed for user `ca84f75d8aa74efdb1e7d791cf5b9488`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:12.718065 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-cdbef572-2a5d-4a8f-bc17-fd835aae421e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:12.788067 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1397/2782] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:12.792765 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9d04c-46ce-4ea1-9de4-da82659afcb0 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:12.858730 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68a9d04c-46ce-4ea1-9de4-da82659afcb0 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:12.863802 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5566f-dc7b-44ca-8457-0f320ea7c313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:12.864522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5566f-dc7b-44ca-8457-0f320ea7c313 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:12.864806 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5566f-dc7b-44ca-8457-0f320ea7c313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:12.865084 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aff5566f-dc7b-44ca-8457-0f320ea7c313 admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:12.865773 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5566f-dc7b-44ca-8457-0f320ea7c313 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:12.867759 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aff5566f-dc7b-44ca-8457-0f320ea7c313 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:12.896166 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1386/2783] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:12 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:12.900293 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbef572-2a5d-4a8f-bc17-fd835aae421e None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:12.911084 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdbef572-2a5d-4a8f-bc17-fd835aae421e None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:12.955459 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:12.970845 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bae003-4780-4121-bb6f-c479cb00d90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:12.971576 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bae003-4780-4121-bb6f-c479cb00d90e admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:12.971901 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bae003-4780-4121-bb6f-c479cb00d90e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:12.972173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59bae003-4780-4121-bb6f-c479cb00d90e admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:12.972888 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bae003-4780-4121-bb6f-c479cb00d90e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:12.974833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59bae003-4780-4121-bb6f-c479cb00d90e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:12.998689 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1398/2784] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:12 2022] POST /identity/v3/users => generated 301 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:13.003603 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5566f-dc7b-44ca-8457-0f320ea7c313 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:13.020257 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aff5566f-dc7b-44ca-8457-0f320ea7c313 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:13.027674 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06c2af-ee3c-49b5-a60f-32a8d6b8b198 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:13.028543 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06c2af-ee3c-49b5-a60f-32a8d6b8b198 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:13.028794 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06c2af-ee3c-49b5-a60f-32a8d6b8b198 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:13.029165 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd06c2af-ee3c-49b5-a60f-32a8d6b8b198 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:13.030079 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06c2af-ee3c-49b5-a60f-32a8d6b8b198 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:13.031838 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd06c2af-ee3c-49b5-a60f-32a8d6b8b198 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:13.042379 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1387/2785] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:13.046533 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bae003-4780-4121-bb6f-c479cb00d90e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:13.056972 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59bae003-4780-4121-bb6f-c479cb00d90e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:13.062135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f2d15-f0ec-4543-ab79-eda1a217c409 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:13.063032 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f2d15-f0ec-4543-ab79-eda1a217c409 admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:13.063322 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f2d15-f0ec-4543-ab79-eda1a217c409 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:13.063610 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057f2d15-f0ec-4543-ab79-eda1a217c409 admin admin] PATH_INFO: `/v3/projects/0434eaee4e8a4113b6c31d8e7f4f37d6/users/56f3cdff793a4b74b90fe2337e394c6b/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:13.079380 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f2d15-f0ec-4543-ab79-eda1a217c409 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0434eaee4e8a4113b6c31d8e7f4f37d6, user_id=56f3cdff793a4b74b90fe2337e394c6b, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:13.082060 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057f2d15-f0ec-4543-ab79-eda1a217c409 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:13.097698 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1399/2786] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:13 2022] PUT /identity/v3/projects/0434eaee4e8a4113b6c31d8e7f4f37d6/users/56f3cdff793a4b74b90fe2337e394c6b/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:16:13.101721 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06c2af-ee3c-49b5-a60f-32a8d6b8b198 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:13.115082 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd06c2af-ee3c-49b5-a60f-32a8d6b8b198 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:13.172287 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdede1-7e3b-400e-a746-c2a476fa5e86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:13.172964 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdede1-7e3b-400e-a746-c2a476fa5e86 admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:13.173256 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdede1-7e3b-400e-a746-c2a476fa5e86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:13.173536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdede1-7e3b-400e-a746-c2a476fa5e86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:13.174279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdede1-7e3b-400e-a746-c2a476fa5e86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:13.175943 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccbdede1-7e3b-400e-a746-c2a476fa5e86 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:13.186186 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1388/2787] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:13 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:13.190422 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f2d15-f0ec-4543-ab79-eda1a217c409 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:13.200798 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057f2d15-f0ec-4543-ab79-eda1a217c409 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:13.205716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ded40-5c7d-4748-bc7d-27d6eaf4981e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:13.206612 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ded40-5c7d-4748-bc7d-27d6eaf4981e admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:13.206898 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ded40-5c7d-4748-bc7d-27d6eaf4981e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:13.207186 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a72ded40-5c7d-4748-bc7d-27d6eaf4981e admin admin] PATH_INFO: `/v3/projects/0434eaee4e8a4113b6c31d8e7f4f37d6/users/56f3cdff793a4b74b90fe2337e394c6b/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:13.209235 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ded40-5c7d-4748-bc7d-27d6eaf4981e admin admin] RBAC: Authorizing `identity:create_grant(project_id=0434eaee4e8a4113b6c31d8e7f4f37d6, user_id=56f3cdff793a4b74b90fe2337e394c6b, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:13.211936 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a72ded40-5c7d-4748-bc7d-27d6eaf4981e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:13.222777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1400/2788] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:13 2022] PUT /identity/v3/projects/0434eaee4e8a4113b6c31d8e7f4f37d6/users/56f3cdff793a4b74b90fe2337e394c6b/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:13.232893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4938c-f3df-4ef8-b49a-17a622b32466 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:13.233184 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4938c-f3df-4ef8-b49a-17a622b32466 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:13.233513 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f4938c-f3df-4ef8-b49a-17a622b32466 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:13.240109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:13.270609 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-18f4938c-f3df-4ef8-b49a-17a622b32466 None None] MFA Rules not processed for user `56f3cdff793a4b74b90fe2337e394c6b`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:13.271808 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-18f4938c-f3df-4ef8-b49a-17a622b32466 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:13.341787 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1389/2789] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:13 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:13.352558 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ded40-5c7d-4748-bc7d-27d6eaf4981e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:13.354316 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-a72ded40-5c7d-4748-bc7d-27d6eaf4981e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:13.419212 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a72ded40-5c7d-4748-bc7d-27d6eaf4981e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:13.424033 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe7e0c0f-ea2a-4ce3-9f34-e596ad3c78bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:13.424725 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e0c0f-ea2a-4ce3-9f34-e596ad3c78bd service cinder] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:13.425038 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e0c0f-ea2a-4ce3-9f34-e596ad3c78bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:13.425319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe7e0c0f-ea2a-4ce3-9f34-e596ad3c78bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:13.435382 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e0c0f-ea2a-4ce3-9f34-e596ad3c78bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:13.436917 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe7e0c0f-ea2a-4ce3-9f34-e596ad3c78bd service cinder] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:13.446608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1401/2790] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:13 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:13.581771 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4938c-f3df-4ef8-b49a-17a622b32466 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:13.660520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f4938c-f3df-4ef8-b49a-17a622b32466 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:13.665581 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933b2f5-7834-435b-8fa0-051c490e061a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:13.666474 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933b2f5-7834-435b-8fa0-051c490e061a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:13.666769 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933b2f5-7834-435b-8fa0-051c490e061a admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:13.667070 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1933b2f5-7834-435b-8fa0-051c490e061a admin admin] PATH_INFO: `/v3/users/ca84f75d8aa74efdb1e7d791cf5b9488` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:13.668354 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933b2f5-7834-435b-8fa0-051c490e061a admin admin] RBAC: Authorizing `identity:delete_user(user_id=ca84f75d8aa74efdb1e7d791cf5b9488)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:13.670370 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1933b2f5-7834-435b-8fa0-051c490e061a admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:13.722688 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1933b2f5-7834-435b-8fa0-051c490e061a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ca84f75d8aa74efdb1e7d791cf5b9488'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:13.735204 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1933b2f5-7834-435b-8fa0-051c490e061a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ca84f75d8aa74efdb1e7d791cf5b9488'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:13.744917 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1933b2f5-7834-435b-8fa0-051c490e061a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ca84f75d8aa74efdb1e7d791cf5b9488'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:13.745471 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1933b2f5-7834-435b-8fa0-051c490e061a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ca84f75d8aa74efdb1e7d791cf5b9488'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:13.752495 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1933b2f5-7834-435b-8fa0-051c490e061a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ca84f75d8aa74efdb1e7d791cf5b9488'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:13.758694 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1390/2791] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:16:13 2022] DELETE /identity/v3/users/ca84f75d8aa74efdb1e7d791cf5b9488 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:13.763546 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4ad36-56fc-46f2-830e-eaa875613dea None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:13.763850 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4ad36-56fc-46f2-830e-eaa875613dea None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:13.764162 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb4ad36-56fc-46f2-830e-eaa875613dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:13.781813 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-bcb4ad36-56fc-46f2-830e-eaa875613dea None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:13.783000 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-bcb4ad36-56fc-46f2-830e-eaa875613dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:13.848520 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1402/2792] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:13.858378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933b2f5-7834-435b-8fa0-051c490e061a admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:13.859950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1933b2f5-7834-435b-8fa0-051c490e061a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:13.870978 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:13.922335 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1933b2f5-7834-435b-8fa0-051c490e061a admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:13.926390 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82319caf-a1a6-47fd-be9c-21ff381e490f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:13.927048 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82319caf-a1a6-47fd-be9c-21ff381e490f service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:13.927335 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82319caf-a1a6-47fd-be9c-21ff381e490f service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:13.927618 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82319caf-a1a6-47fd-be9c-21ff381e490f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:13.937258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82319caf-a1a6-47fd-be9c-21ff381e490f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:13.938719 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82319caf-a1a6-47fd-be9c-21ff381e490f service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:13.948433 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1391/2793] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:14.232038 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4ad36-56fc-46f2-830e-eaa875613dea None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:14.233523 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-bcb4ad36-56fc-46f2-830e-eaa875613dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:14.248729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb4ad36-56fc-46f2-830e-eaa875613dea None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:14.253527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:14.254413 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:14.255044 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:14.255318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4 admin admin] PATH_INFO: `/v3/projects/fa382782425d44f9a654480c1a2794c3` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:14.256519 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa382782425d44f9a654480c1a2794c3)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:14.258420 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:14.305008 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:14.309307 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa382782425d44f9a654480c1a2794c3', 'request_id': 'req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:14.315544 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa382782425d44f9a654480c1a2794c3', 'request_id': 'req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:14.324986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa382782425d44f9a654480c1a2794c3', 'request_id': 'req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:14.336690 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4 admin admin] The token cache is being invalidate because project fa382782425d44f9a654480c1a2794c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:14.337090 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:14.337735 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:14.339322 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1403/2794] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:14 2022] DELETE /identity/v3/projects/fa382782425d44f9a654480c1a2794c3 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:14.343729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82319caf-a1a6-47fd-be9c-21ff381e490f service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:14.353810 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82319caf-a1a6-47fd-be9c-21ff381e490f service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:14.418857 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fababdfa-e02f-4c06-b1c2-d1065dd24849 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:14.419783 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fababdfa-e02f-4c06-b1c2-d1065dd24849 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:14.420107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fababdfa-e02f-4c06-b1c2-d1065dd24849 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:14.420389 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fababdfa-e02f-4c06-b1c2-d1065dd24849 admin admin] PATH_INFO: `/v3/users/56f3cdff793a4b74b90fe2337e394c6b` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:14.421636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fababdfa-e02f-4c06-b1c2-d1065dd24849 admin admin] RBAC: Authorizing `identity:delete_user(user_id=56f3cdff793a4b74b90fe2337e394c6b)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:14.423626 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fababdfa-e02f-4c06-b1c2-d1065dd24849 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:14.471345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fababdfa-e02f-4c06-b1c2-d1065dd24849 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '56f3cdff793a4b74b90fe2337e394c6b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:14.484441 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fababdfa-e02f-4c06-b1c2-d1065dd24849 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '56f3cdff793a4b74b90fe2337e394c6b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:14.494220 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fababdfa-e02f-4c06-b1c2-d1065dd24849 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '56f3cdff793a4b74b90fe2337e394c6b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:14.494862 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fababdfa-e02f-4c06-b1c2-d1065dd24849 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '56f3cdff793a4b74b90fe2337e394c6b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:14.501987 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-fababdfa-e02f-4c06-b1c2-d1065dd24849 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '56f3cdff793a4b74b90fe2337e394c6b'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:14.508292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1392/2795] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:16:14 2022] DELETE /identity/v3/users/56f3cdff793a4b74b90fe2337e394c6b => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:14.795180 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:14.796765 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:14.811662 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0929fb-6ef5-4bc7-8f56-077d48679cc4 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:14.873801 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2ccba6-500b-4e0d-be47-7c3a09b0b2f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:14.874729 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ccba6-500b-4e0d-be47-7c3a09b0b2f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:14.875025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ccba6-500b-4e0d-be47-7c3a09b0b2f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:14.875302 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2ccba6-500b-4e0d-be47-7c3a09b0b2f0 admin admin] PATH_INFO: `/v3/projects/0434eaee4e8a4113b6c31d8e7f4f37d6` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:14.876581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ccba6-500b-4e0d-be47-7c3a09b0b2f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0434eaee4e8a4113b6c31d8e7f4f37d6)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:14.878573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2ccba6-500b-4e0d-be47-7c3a09b0b2f0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:14.927958 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8c2ccba6-500b-4e0d-be47-7c3a09b0b2f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0434eaee4e8a4113b6c31d8e7f4f37d6', 'request_id': 'req-8c2ccba6-500b-4e0d-be47-7c3a09b0b2f0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:14.934116 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8c2ccba6-500b-4e0d-be47-7c3a09b0b2f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0434eaee4e8a4113b6c31d8e7f4f37d6', 'request_id': 'req-8c2ccba6-500b-4e0d-be47-7c3a09b0b2f0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:14.942846 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8c2ccba6-500b-4e0d-be47-7c3a09b0b2f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0434eaee4e8a4113b6c31d8e7f4f37d6', 'request_id': 'req-8c2ccba6-500b-4e0d-be47-7c3a09b0b2f0'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:14.953969 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8c2ccba6-500b-4e0d-be47-7c3a09b0b2f0 admin admin] The token cache is being invalidate because project 0434eaee4e8a4113b6c31d8e7f4f37d6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:14.954264 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8c2ccba6-500b-4e0d-be47-7c3a09b0b2f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:14.954889 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-8c2ccba6-500b-4e0d-be47-7c3a09b0b2f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:14.956482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1404/2796] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:14 2022] DELETE /identity/v3/projects/0434eaee4e8a4113b6c31d8e7f4f37d6 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:14.963082 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36aff2-91c5-46fc-ac77-b1867b58c51d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:14.963379 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36aff2-91c5-46fc-ac77-b1867b58c51d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:14.963663 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f36aff2-91c5-46fc-ac77-b1867b58c51d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:14.982533 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9f36aff2-91c5-46fc-ac77-b1867b58c51d None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:14.983739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9f36aff2-91c5-46fc-ac77-b1867b58c51d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:15.017373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:15.059274 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1393/2797] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:15.068195 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b625604-fbbe-4deb-a348-b49769c72927 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:15.068493 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b625604-fbbe-4deb-a348-b49769c72927 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:15.068780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b625604-fbbe-4deb-a348-b49769c72927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:15.087981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-3b625604-fbbe-4deb-a348-b49769c72927 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:15.089319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3b625604-fbbe-4deb-a348-b49769c72927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:15.098411 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1405/2798] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:15.102695 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36aff2-91c5-46fc-ac77-b1867b58c51d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:15.112963 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f36aff2-91c5-46fc-ac77-b1867b58c51d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:15.118540 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5854652-720b-49b2-800c-8e401517221e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:15.119209 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5854652-720b-49b2-800c-8e401517221e admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:15.119503 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5854652-720b-49b2-800c-8e401517221e admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:15.119787 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5854652-720b-49b2-800c-8e401517221e admin admin] PATH_INFO: `/v3/domains` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:15.120565 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5854652-720b-49b2-800c-8e401517221e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80233) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:16:15.120877 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5854652-720b-49b2-800c-8e401517221e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:15.122522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5854652-720b-49b2-800c-8e401517221e admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:15.135281 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1394/2799] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:16:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:16:15.139811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b625604-fbbe-4deb-a348-b49769c72927 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:15.149381 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b625604-fbbe-4deb-a348-b49769c72927 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:15.158561 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b76a5-b4db-4106-85be-2cc6b276b281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:15.159266 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b76a5-b4db-4106-85be-2cc6b276b281 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:15.159559 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b76a5-b4db-4106-85be-2cc6b276b281 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:15.159907 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537b76a5-b4db-4106-85be-2cc6b276b281 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:15.160711 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b76a5-b4db-4106-85be-2cc6b276b281 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:15.162736 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537b76a5-b4db-4106-85be-2cc6b276b281 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:15.194902 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1406/2800] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:15 2022] POST /identity/v3/projects => generated 385 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:15.199384 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5854652-720b-49b2-800c-8e401517221e admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:15.210139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5854652-720b-49b2-800c-8e401517221e admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:15.268643 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61aeca0-7f6b-4483-9934-386f12485fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:15.269296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61aeca0-7f6b-4483-9934-386f12485fbe admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:15.269624 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61aeca0-7f6b-4483-9934-386f12485fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:15.269911 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61aeca0-7f6b-4483-9934-386f12485fbe admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:15.270705 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61aeca0-7f6b-4483-9934-386f12485fbe admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:15.273301 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61aeca0-7f6b-4483-9934-386f12485fbe admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:15.309484 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1395/2801] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:15 2022] POST /identity/v3/users => generated 293 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:15.313818 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b76a5-b4db-4106-85be-2cc6b276b281 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:15.324632 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537b76a5-b4db-4106-85be-2cc6b276b281 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:15.329279 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d6baf-e7bf-441d-a414-a81314b7214c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:15.330028 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d6baf-e7bf-441d-a414-a81314b7214c admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:15.330221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d6baf-e7bf-441d-a414-a81314b7214c admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:15.330512 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a6d6baf-e7bf-441d-a414-a81314b7214c admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:15.331275 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d6baf-e7bf-441d-a414-a81314b7214c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:15.332890 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a6d6baf-e7bf-441d-a414-a81314b7214c admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:15.345161 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1407/2802] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:15.349241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61aeca0-7f6b-4483-9934-386f12485fbe admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:15.359183 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61aeca0-7f6b-4483-9934-386f12485fbe admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:15.363927 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e88faf4-fa27-45ca-b730-0d34b9ca556f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:15.364585 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88faf4-fa27-45ca-b730-0d34b9ca556f admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:15.364869 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88faf4-fa27-45ca-b730-0d34b9ca556f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:15.365149 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e88faf4-fa27-45ca-b730-0d34b9ca556f admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:15.365891 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88faf4-fa27-45ca-b730-0d34b9ca556f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:15.367522 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e88faf4-fa27-45ca-b730-0d34b9ca556f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:15.377353 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1396/2803] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:15 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:15.383256 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d6baf-e7bf-441d-a414-a81314b7214c admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:15.394103 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a6d6baf-e7bf-441d-a414-a81314b7214c admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:15.399723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc6c45-7677-4a06-9520-7f15e2e0bfae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:15.400685 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc6c45-7677-4a06-9520-7f15e2e0bfae admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:15.400975 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc6c45-7677-4a06-9520-7f15e2e0bfae admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:15.401257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbc6c45-7677-4a06-9520-7f15e2e0bfae admin admin] PATH_INFO: `/v3/projects/e2fbd40b4ea447319a8b90656548da9e/users/920a00aa201143f6ae4799aff51c6dae/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:15.413723 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:15.417406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc6c45-7677-4a06-9520-7f15e2e0bfae admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2fbd40b4ea447319a8b90656548da9e, user_id=920a00aa201143f6ae4799aff51c6dae, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:15.421248 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbc6c45-7677-4a06-9520-7f15e2e0bfae admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:15.437677 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1408/2804] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:15 2022] PUT /identity/v3/projects/e2fbd40b4ea447319a8b90656548da9e/users/920a00aa201143f6ae4799aff51c6dae/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:15.442855 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799263fa-0965-46c1-a71e-0f857cb6d985 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:15.443159 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799263fa-0965-46c1-a71e-0f857cb6d985 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:15.443457 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799263fa-0965-46c1-a71e-0f857cb6d985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:15.461456 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-799263fa-0965-46c1-a71e-0f857cb6d985 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:15.462673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-799263fa-0965-46c1-a71e-0f857cb6d985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:15.525586 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1397/2805] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:15.535775 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc6c45-7677-4a06-9520-7f15e2e0bfae admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:15.537721 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-fdbc6c45-7677-4a06-9520-7f15e2e0bfae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:15.596648 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbc6c45-7677-4a06-9520-7f15e2e0bfae admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:15.600601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b3bd2-7435-47f2-87b8-c83cae7698e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:15.601541 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b3bd2-7435-47f2-87b8-c83cae7698e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:15.601541 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b3bd2-7435-47f2-87b8-c83cae7698e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:15.601784 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122b3bd2-7435-47f2-87b8-c83cae7698e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:15.611451 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b3bd2-7435-47f2-87b8-c83cae7698e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:15.612963 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-122b3bd2-7435-47f2-87b8-c83cae7698e9 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:15.623122 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1409/2806] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:19.854597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205a904-f29e-4a1d-ae90-43c426a03c5d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:19.854597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205a904-f29e-4a1d-ae90-43c426a03c5d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:19.854597 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5205a904-f29e-4a1d-ae90-43c426a03c5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:19.885274 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-5205a904-f29e-4a1d-ae90-43c426a03c5d None None] MFA Rules not processed for user `920a00aa201143f6ae4799aff51c6dae`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:19.886796 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5205a904-f29e-4a1d-ae90-43c426a03c5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:19.915744 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:19.976954 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1398/2807] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:19 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:19.981587 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b3bd2-7435-47f2-87b8-c83cae7698e9 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:20.000355 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122b3bd2-7435-47f2-87b8-c83cae7698e9 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:20.005955 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051b4c0-06a1-4239-99de-ebad5ba00088 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:20.006986 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051b4c0-06a1-4239-99de-ebad5ba00088 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:20.007271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051b4c0-06a1-4239-99de-ebad5ba00088 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:20.007772 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7051b4c0-06a1-4239-99de-ebad5ba00088 admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:20.008537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051b4c0-06a1-4239-99de-ebad5ba00088 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:20.010474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7051b4c0-06a1-4239-99de-ebad5ba00088 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:20.042968 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1410/2808] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:19 2022] POST /identity/v3/projects => generated 383 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:20.046839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205a904-f29e-4a1d-ae90-43c426a03c5d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:20.059425 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5205a904-f29e-4a1d-ae90-43c426a03c5d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:20.124233 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e786c13-0075-4a04-a4c4-609e40f3b6d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:20.124931 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e786c13-0075-4a04-a4c4-609e40f3b6d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:20.125222 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e786c13-0075-4a04-a4c4-609e40f3b6d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:20.125510 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e786c13-0075-4a04-a4c4-609e40f3b6d4 admin admin] PATH_INFO: `/v3/users` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:20.126276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e786c13-0075-4a04-a4c4-609e40f3b6d4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:20.128279 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e786c13-0075-4a04-a4c4-609e40f3b6d4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:20.154607 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1399/2809] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:20 2022] POST /identity/v3/users => generated 298 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:20.155977 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051b4c0-06a1-4239-99de-ebad5ba00088 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:20.167345 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7051b4c0-06a1-4239-99de-ebad5ba00088 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:20.172584 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8604b5d-5154-441c-8ac4-2b830cf54122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:20.173436 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8604b5d-5154-441c-8ac4-2b830cf54122 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:20.173982 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8604b5d-5154-441c-8ac4-2b830cf54122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:20.174335 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8604b5d-5154-441c-8ac4-2b830cf54122 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:20.175192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8604b5d-5154-441c-8ac4-2b830cf54122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:20.176977 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8604b5d-5154-441c-8ac4-2b830cf54122 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:20.190213 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1411/2810] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:20.194660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e786c13-0075-4a04-a4c4-609e40f3b6d4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:20.205674 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e786c13-0075-4a04-a4c4-609e40f3b6d4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:20.210701 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d8467-84ab-4ed3-a8e8-416770e7315c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:20.211722 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d8467-84ab-4ed3-a8e8-416770e7315c admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:20.212119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d8467-84ab-4ed3-a8e8-416770e7315c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:20.212464 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b30d8467-84ab-4ed3-a8e8-416770e7315c admin admin] PATH_INFO: `/v3/projects/22e9ac6dbb5149379466be8e688c58bf/users/1b008d0814014b979f86ddfebc1e4e24/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:20.228998 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d8467-84ab-4ed3-a8e8-416770e7315c admin admin] RBAC: Authorizing `identity:create_grant(project_id=22e9ac6dbb5149379466be8e688c58bf, user_id=1b008d0814014b979f86ddfebc1e4e24, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:20.231581 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b30d8467-84ab-4ed3-a8e8-416770e7315c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:20.246600 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1400/2811] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:20 2022] PUT /identity/v3/projects/22e9ac6dbb5149379466be8e688c58bf/users/1b008d0814014b979f86ddfebc1e4e24/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:20.250605 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8604b5d-5154-441c-8ac4-2b830cf54122 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:20.261236 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8604b5d-5154-441c-8ac4-2b830cf54122 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:20.324204 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aafddde-f963-41ac-9bc8-95963c395861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:20.324898 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafddde-f963-41ac-9bc8-95963c395861 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:20.325192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafddde-f963-41ac-9bc8-95963c395861 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:20.325479 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aafddde-f963-41ac-9bc8-95963c395861 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:20.326222 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aafddde-f963-41ac-9bc8-95963c395861 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:20.327877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9aafddde-f963-41ac-9bc8-95963c395861 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:20.338533 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1412/2812] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:20 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:20.342747 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d8467-84ab-4ed3-a8e8-416770e7315c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:20.353909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b30d8467-84ab-4ed3-a8e8-416770e7315c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:20.358731 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665035ae-2b2c-4768-8dde-4c9fc5ce1ff6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:20.359672 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665035ae-2b2c-4768-8dde-4c9fc5ce1ff6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:20.360009 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665035ae-2b2c-4768-8dde-4c9fc5ce1ff6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:20.360296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-665035ae-2b2c-4768-8dde-4c9fc5ce1ff6 admin admin] PATH_INFO: `/v3/projects/22e9ac6dbb5149379466be8e688c58bf/users/1b008d0814014b979f86ddfebc1e4e24/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:20.362515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665035ae-2b2c-4768-8dde-4c9fc5ce1ff6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=22e9ac6dbb5149379466be8e688c58bf, user_id=1b008d0814014b979f86ddfebc1e4e24, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:20.365258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-665035ae-2b2c-4768-8dde-4c9fc5ce1ff6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:20.377036 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1401/2813] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:20 2022] PUT /identity/v3/projects/22e9ac6dbb5149379466be8e688c58bf/users/1b008d0814014b979f86ddfebc1e4e24/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:24.805753 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276fd2f-eb97-4b5f-af4e-000a3798a73d None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:24.806291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276fd2f-eb97-4b5f-af4e-000a3798a73d None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:24.806393 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3276fd2f-eb97-4b5f-af4e-000a3798a73d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:24.838270 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-3276fd2f-eb97-4b5f-af4e-000a3798a73d None None] MFA Rules not processed for user `1b008d0814014b979f86ddfebc1e4e24`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:24.839630 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3276fd2f-eb97-4b5f-af4e-000a3798a73d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:24.896190 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:24.928471 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1413/2814] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:24 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:24.940743 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665035ae-2b2c-4768-8dde-4c9fc5ce1ff6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:24.949487 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-665035ae-2b2c-4768-8dde-4c9fc5ce1ff6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:25.020636 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-665035ae-2b2c-4768-8dde-4c9fc5ce1ff6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:25.024486 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf0a75-00b9-4766-b155-9c21de409caa service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '196dc406a95444c39babc50d59f9eb6b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:25.025158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf0a75-00b9-4766-b155-9c21de409caa service glance] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:25.025451 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf0a75-00b9-4766-b155-9c21de409caa service glance] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:25.025740 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdf0a75-00b9-4766-b155-9c21de409caa service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:25.061222 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:25.100081 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf0a75-00b9-4766-b155-9c21de409caa service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:25.101577 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdf0a75-00b9-4766-b155-9c21de409caa service glance] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:25.114496 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1402/2815] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:24 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 175 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:25.255209 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc1879-3f6d-417f-9e79-615077fc8de2 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:25.255537 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc1879-3f6d-417f-9e79-615077fc8de2 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:25.255936 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dc1879-3f6d-417f-9e79-615077fc8de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:25.285903 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-32dc1879-3f6d-417f-9e79-615077fc8de2 None None] MFA Rules not processed for user `066de1f11f724eee9df0377beb58d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:25.285903 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-32dc1879-3f6d-417f-9e79-615077fc8de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:25.355484 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1414/2816] 10.222.0.85 () {58 vars in 1009 bytes} [Wed May 25 16:16:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:25.420815 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf0a75-00b9-4766-b155-9c21de409caa service glance] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:25.422293 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cfdf0a75-00b9-4766-b155-9c21de409caa service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:25.498924 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdf0a75-00b9-4766-b155-9c21de409caa service glance] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:25.504404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05ad9cfe-02ef-4474-88b3-07529010c8ac service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77fda27641c84fccb68a5cdaf4e8dc4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:25.505080 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad9cfe-02ef-4474-88b3-07529010c8ac service swift] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:25.505367 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad9cfe-02ef-4474-88b3-07529010c8ac service swift] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:25.505646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05ad9cfe-02ef-4474-88b3-07529010c8ac service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:25.515477 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad9cfe-02ef-4474-88b3-07529010c8ac service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:25.517154 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05ad9cfe-02ef-4474-88b3-07529010c8ac service swift] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:25.528061 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1403/2817] 10.222.0.85 () {60 vars in 1384 bytes} [Wed May 25 16:16:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 16:16:31.416170 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc1879-3f6d-417f-9e79-615077fc8de2 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:31.417632 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-32dc1879-3f6d-417f-9e79-615077fc8de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:31.491930 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dc1879-3f6d-417f-9e79-615077fc8de2 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:31.496720 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edede8cc-6383-4afd-93dd-f183f37a756d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:31.497786 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edede8cc-6383-4afd-93dd-f183f37a756d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:31.498243 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edede8cc-6383-4afd-93dd-f183f37a756d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:31.498680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edede8cc-6383-4afd-93dd-f183f37a756d admin admin] PATH_INFO: `/v3/users/4e864a9b49f24af9b7f517ddddd01776` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:31.502424 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edede8cc-6383-4afd-93dd-f183f37a756d admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e864a9b49f24af9b7f517ddddd01776)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:31.504446 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edede8cc-6383-4afd-93dd-f183f37a756d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:31.579134 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-edede8cc-6383-4afd-93dd-f183f37a756d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e864a9b49f24af9b7f517ddddd01776'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:31.579881 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-edede8cc-6383-4afd-93dd-f183f37a756d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e864a9b49f24af9b7f517ddddd01776'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:31.593890 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-edede8cc-6383-4afd-93dd-f183f37a756d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e864a9b49f24af9b7f517ddddd01776'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:31.603353 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-edede8cc-6383-4afd-93dd-f183f37a756d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e864a9b49f24af9b7f517ddddd01776'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:31.603997 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-edede8cc-6383-4afd-93dd-f183f37a756d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e864a9b49f24af9b7f517ddddd01776'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:31.620621 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1415/2818] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:16:31 2022] DELETE /identity/v3/users/4e864a9b49f24af9b7f517ddddd01776 => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:31.624941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec123c8d-2f4f-4fed-af7a-27be2e0c0cf7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:31.625488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec123c8d-2f4f-4fed-af7a-27be2e0c0cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:31.625692 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec123c8d-2f4f-4fed-af7a-27be2e0c0cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:31.644153 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ec123c8d-2f4f-4fed-af7a-27be2e0c0cf7 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:31.647682 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ec123c8d-2f4f-4fed-af7a-27be2e0c0cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:31.721624 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1404/2819] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:31.732230 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edede8cc-6383-4afd-93dd-f183f37a756d admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:31.733607 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-edede8cc-6383-4afd-93dd-f183f37a756d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:31.743970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:31.799989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edede8cc-6383-4afd-93dd-f183f37a756d admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:31.803591 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfe5cbd6-887f-494f-98c2-cb8c915ce061 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:31.804292 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5cbd6-887f-494f-98c2-cb8c915ce061 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:31.804578 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5cbd6-887f-494f-98c2-cb8c915ce061 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:31.804880 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfe5cbd6-887f-494f-98c2-cb8c915ce061 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:31.817716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5cbd6-887f-494f-98c2-cb8c915ce061 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:31.819466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfe5cbd6-887f-494f-98c2-cb8c915ce061 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:31.831067 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1416/2820] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:32.141190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec123c8d-2f4f-4fed-af7a-27be2e0c0cf7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:32.142642 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ec123c8d-2f4f-4fed-af7a-27be2e0c0cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:32.158871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec123c8d-2f4f-4fed-af7a-27be2e0c0cf7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:32.165976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0632a4de-35b8-4b42-a0a2-0f775675b5ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:32.166897 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632a4de-35b8-4b42-a0a2-0f775675b5ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:32.167173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632a4de-35b8-4b42-a0a2-0f775675b5ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:32.167456 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0632a4de-35b8-4b42-a0a2-0f775675b5ce admin admin] PATH_INFO: `/v3/projects/852905969b234e5386566425166c1f7f` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:32.168695 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0632a4de-35b8-4b42-a0a2-0f775675b5ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=852905969b234e5386566425166c1f7f)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:32.170913 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0632a4de-35b8-4b42-a0a2-0f775675b5ce admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:32.234558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0632a4de-35b8-4b42-a0a2-0f775675b5ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '852905969b234e5386566425166c1f7f', 'request_id': 'req-0632a4de-35b8-4b42-a0a2-0f775675b5ce'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:32.248234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0632a4de-35b8-4b42-a0a2-0f775675b5ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '852905969b234e5386566425166c1f7f', 'request_id': 'req-0632a4de-35b8-4b42-a0a2-0f775675b5ce'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:32.250131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:32.259551 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0632a4de-35b8-4b42-a0a2-0f775675b5ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '852905969b234e5386566425166c1f7f', 'request_id': 'req-0632a4de-35b8-4b42-a0a2-0f775675b5ce'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:32.274068 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0632a4de-35b8-4b42-a0a2-0f775675b5ce admin admin] The token cache is being invalidate because project 852905969b234e5386566425166c1f7f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:32.274373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0632a4de-35b8-4b42-a0a2-0f775675b5ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:32.275037 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-0632a4de-35b8-4b42-a0a2-0f775675b5ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:32.279421 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1405/2821] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:32 2022] DELETE /identity/v3/projects/852905969b234e5386566425166c1f7f => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:32.284495 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7759aba-72af-4f71-a74f-55aba085b326 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:32.284791 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7759aba-72af-4f71-a74f-55aba085b326 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:32.285073 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7759aba-72af-4f71-a74f-55aba085b326 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:32.305934 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-c7759aba-72af-4f71-a74f-55aba085b326 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:32.307175 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-c7759aba-72af-4f71-a74f-55aba085b326 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:32.377271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1417/2822] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:32.385466 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082fd4-415e-4354-bd00-fc7b1fc1a3db None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:32.385770 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082fd4-415e-4354-bd00-fc7b1fc1a3db None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:32.386060 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca082fd4-415e-4354-bd00-fc7b1fc1a3db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:32.406243 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ca082fd4-415e-4354-bd00-fc7b1fc1a3db None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:32.410069 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ca082fd4-415e-4354-bd00-fc7b1fc1a3db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:32.419860 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1406/2823] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:32.422831 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7759aba-72af-4f71-a74f-55aba085b326 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:32.435019 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7759aba-72af-4f71-a74f-55aba085b326 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:32.440182 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a42fb5-2384-4753-9f30-6c2bffafb5b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:32.440874 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a42fb5-2384-4753-9f30-6c2bffafb5b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:32.441169 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a42fb5-2384-4753-9f30-6c2bffafb5b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:32.441450 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a42fb5-2384-4753-9f30-6c2bffafb5b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:32.442221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a42fb5-2384-4753-9f30-6c2bffafb5b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:16:32.442577 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a42fb5-2384-4753-9f30-6c2bffafb5b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:32.444245 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a42fb5-2384-4753-9f30-6c2bffafb5b3 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:32.459873 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1418/2824] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:16:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:16:32.463086 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca082fd4-415e-4354-bd00-fc7b1fc1a3db None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:32.473769 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca082fd4-415e-4354-bd00-fc7b1fc1a3db None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:32.482784 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f20fc-3067-491e-abde-49e52edade3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:32.483558 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f20fc-3067-491e-abde-49e52edade3f admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:32.483950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f20fc-3067-491e-abde-49e52edade3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:32.484309 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76f20fc-3067-491e-abde-49e52edade3f admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:32.485096 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f20fc-3067-491e-abde-49e52edade3f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:32.489729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c76f20fc-3067-491e-abde-49e52edade3f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:32.520980 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1407/2825] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:32 2022] POST /identity/v3/projects => generated 379 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:32.524989 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a42fb5-2384-4753-9f30-6c2bffafb5b3 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:32.538494 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a42fb5-2384-4753-9f30-6c2bffafb5b3 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:32.653214 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f07aa-e725-4bf2-80f0-75d746696b8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:32.653964 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f07aa-e725-4bf2-80f0-75d746696b8a admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:32.654269 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f07aa-e725-4bf2-80f0-75d746696b8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:32.654546 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8f07aa-e725-4bf2-80f0-75d746696b8a admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:32.655442 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f07aa-e725-4bf2-80f0-75d746696b8a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:32.657461 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8f07aa-e725-4bf2-80f0-75d746696b8a admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:32.692651 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1419/2826] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:32 2022] POST /identity/v3/users => generated 290 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:32.693902 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f20fc-3067-491e-abde-49e52edade3f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:32.706091 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76f20fc-3067-491e-abde-49e52edade3f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:32.713742 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78caea-5c9f-4d25-81b1-6d794f5a1dfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:32.715066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78caea-5c9f-4d25-81b1-6d794f5a1dfb admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:32.715645 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78caea-5c9f-4d25-81b1-6d794f5a1dfb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:32.715809 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc78caea-5c9f-4d25-81b1-6d794f5a1dfb admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:32.716519 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78caea-5c9f-4d25-81b1-6d794f5a1dfb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:32.718173 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc78caea-5c9f-4d25-81b1-6d794f5a1dfb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:32.735515 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1408/2827] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:32.737056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f07aa-e725-4bf2-80f0-75d746696b8a admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:32.747898 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8f07aa-e725-4bf2-80f0-75d746696b8a admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:32.755769 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9a123c-e3ad-4250-8766-c1e5538ca9e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:32.756701 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a123c-e3ad-4250-8766-c1e5538ca9e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:32.757054 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a123c-e3ad-4250-8766-c1e5538ca9e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:32.757263 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9a123c-e3ad-4250-8766-c1e5538ca9e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:32.758015 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a123c-e3ad-4250-8766-c1e5538ca9e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:32.759766 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9a123c-e3ad-4250-8766-c1e5538ca9e2 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:32.776057 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1420/2828] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:32 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:32.777673 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78caea-5c9f-4d25-81b1-6d794f5a1dfb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:32.788993 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc78caea-5c9f-4d25-81b1-6d794f5a1dfb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:32.794004 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba861d-4206-488b-a6d1-87039d995d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:32.795138 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba861d-4206-488b-a6d1-87039d995d76 admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:32.795613 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba861d-4206-488b-a6d1-87039d995d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:32.796119 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ba861d-4206-488b-a6d1-87039d995d76 admin admin] PATH_INFO: `/v3/projects/31462b97f7a04859b66398fd6386cfc4/users/67b7bca2e2ef4dbf89554ba2f9bf93e5/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:32.813136 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba861d-4206-488b-a6d1-87039d995d76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=31462b97f7a04859b66398fd6386cfc4, user_id=67b7bca2e2ef4dbf89554ba2f9bf93e5, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:32.816007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ba861d-4206-488b-a6d1-87039d995d76 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:32.835567 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1409/2829] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:32 2022] PUT /identity/v3/projects/31462b97f7a04859b66398fd6386cfc4/users/67b7bca2e2ef4dbf89554ba2f9bf93e5/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:32.840897 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747635fd-b318-4549-848e-b1aceeded2f9 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:32.841405 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747635fd-b318-4549-848e-b1aceeded2f9 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:32.841785 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-747635fd-b318-4549-848e-b1aceeded2f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:32.860114 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-747635fd-b318-4549-848e-b1aceeded2f9 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:32.861442 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-747635fd-b318-4549-848e-b1aceeded2f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:32.873656 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:32.942221 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1421/2830] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:32.952901 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba861d-4206-488b-a6d1-87039d995d76 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:32.954706 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-31ba861d-4206-488b-a6d1-87039d995d76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:33.020500 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ba861d-4206-488b-a6d1-87039d995d76 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:33.024417 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd58d9b9-dfba-4eb3-a407-a4274f125c39 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:33.025229 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58d9b9-dfba-4eb3-a407-a4274f125c39 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:33.025537 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58d9b9-dfba-4eb3-a407-a4274f125c39 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:33.025847 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd58d9b9-dfba-4eb3-a407-a4274f125c39 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:33.036599 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58d9b9-dfba-4eb3-a407-a4274f125c39 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:33.037988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd58d9b9-dfba-4eb3-a407-a4274f125c39 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:33.049413 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1410/2831] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:34.832905 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0c7bc-6973-4e77-8c01-ca4e610a4d59 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:34.833387 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0c7bc-6973-4e77-8c01-ca4e610a4d59 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:34.833501 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3b0c7bc-6973-4e77-8c01-ca4e610a4d59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:34.854143 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-d3b0c7bc-6973-4e77-8c01-ca4e610a4d59 None None] MFA Rules not processed for user `920a00aa201143f6ae4799aff51c6dae`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:34.855579 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d3b0c7bc-6973-4e77-8c01-ca4e610a4d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:34.937821 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1422/2832] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:34 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:37.328988 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0139011-a679-4ae9-8923-04e9d9b6251d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:37.330037 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0139011-a679-4ae9-8923-04e9d9b6251d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:37.330379 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0139011-a679-4ae9-8923-04e9d9b6251d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:37.352951 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-e0139011-a679-4ae9-8923-04e9d9b6251d None None] MFA Rules not processed for user `066de1f11f724eee9df0377beb58d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:37.355199 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e0139011-a679-4ae9-8923-04e9d9b6251d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:37.471214 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1411/2833] 10.222.0.85 () {58 vars in 1009 bytes} [Wed May 25 16:16:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:37.486088 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0c7bc-6973-4e77-8c01-ca4e610a4d59 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:37.488474 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d3b0c7bc-6973-4e77-8c01-ca4e610a4d59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:37.574891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3b0c7bc-6973-4e77-8c01-ca4e610a4d59 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:37.581041 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c73a7-5882-4daf-aaa7-fbfeedc03c54 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77fda27641c84fccb68a5cdaf4e8dc4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:37.581965 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c73a7-5882-4daf-aaa7-fbfeedc03c54 service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:37.582419 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c73a7-5882-4daf-aaa7-fbfeedc03c54 service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:37.582869 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-460c73a7-5882-4daf-aaa7-fbfeedc03c54 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:37.592617 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c73a7-5882-4daf-aaa7-fbfeedc03c54 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:37.594877 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-460c73a7-5882-4daf-aaa7-fbfeedc03c54 service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:37.625908 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1423/2834] 10.222.0.85 () {60 vars in 1384 bytes} [Wed May 25 16:16:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) May 25 16:16:37.675346 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135a74a-ace6-4c72-bd12-88e62cc581a5 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:37.675742 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135a74a-ace6-4c72-bd12-88e62cc581a5 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:37.676274 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1135a74a-ace6-4c72-bd12-88e62cc581a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:37.728910 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-1135a74a-ace6-4c72-bd12-88e62cc581a5 None None] MFA Rules not processed for user `67b7bca2e2ef4dbf89554ba2f9bf93e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:37.730438 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1135a74a-ace6-4c72-bd12-88e62cc581a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:37.770314 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=96,keystone:DELETE=2,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:37.936369 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1412/2835] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:37 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:37.940688 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c73a7-5882-4daf-aaa7-fbfeedc03c54 service swift] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:37.958277 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135a74a-ace6-4c72-bd12-88e62cc581a5 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:37.960870 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1135a74a-ace6-4c72-bd12-88e62cc581a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:37.962458 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-460c73a7-5882-4daf-aaa7-fbfeedc03c54 service swift] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:37.983956 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de308332-43be-4b2f-b664-fc806dee274b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:37.984715 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de308332-43be-4b2f-b664-fc806dee274b admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:37.984998 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de308332-43be-4b2f-b664-fc806dee274b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:37.985363 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de308332-43be-4b2f-b664-fc806dee274b admin admin] PATH_INFO: `/v3/projects` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:37.986082 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de308332-43be-4b2f-b664-fc806dee274b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:37.988149 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de308332-43be-4b2f-b664-fc806dee274b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:38.017234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1135a74a-ace6-4c72-bd12-88e62cc581a5 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:38.029768 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b760a-f8f5-4951-b18f-0dc12c81f704 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:38.031661 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b760a-f8f5-4951-b18f-0dc12c81f704 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:38.031997 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b760a-f8f5-4951-b18f-0dc12c81f704 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:38.032284 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b760a-f8f5-4951-b18f-0dc12c81f704 admin admin] PATH_INFO: `/v3/users/daa5a4286c5e421498bbf8d1c7cdd6b7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:38.038822 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b760a-f8f5-4951-b18f-0dc12c81f704 admin admin] RBAC: Authorizing `identity:delete_user(user_id=daa5a4286c5e421498bbf8d1c7cdd6b7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:38.040946 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b760a-f8f5-4951-b18f-0dc12c81f704 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:38.044390 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1424/2836] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:37 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:38.049123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de308332-43be-4b2f-b664-fc806dee274b admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:38.121808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:38.155720 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-436b760a-f8f5-4951-b18f-0dc12c81f704 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'daa5a4286c5e421498bbf8d1c7cdd6b7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:38.189738 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-436b760a-f8f5-4951-b18f-0dc12c81f704 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'daa5a4286c5e421498bbf8d1c7cdd6b7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:38.191600 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de308332-43be-4b2f-b664-fc806dee274b admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:38.200267 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5a940-8927-4d17-8111-fa39bd5283b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:38.203096 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5a940-8927-4d17-8111-fa39bd5283b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:38.203468 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5a940-8927-4d17-8111-fa39bd5283b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:38.203954 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b5a940-8927-4d17-8111-fa39bd5283b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:38.204482 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5a940-8927-4d17-8111-fa39bd5283b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:38.206706 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b5a940-8927-4d17-8111-fa39bd5283b9 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:38.207797 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-436b760a-f8f5-4951-b18f-0dc12c81f704 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'daa5a4286c5e421498bbf8d1c7cdd6b7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:38.208623 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-436b760a-f8f5-4951-b18f-0dc12c81f704 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'daa5a4286c5e421498bbf8d1c7cdd6b7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:38.257547 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1425/2837] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:38 2022] POST /identity/v3/users => generated 296 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:16:38.258023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-436b760a-f8f5-4951-b18f-0dc12c81f704 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'daa5a4286c5e421498bbf8d1c7cdd6b7'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:38.261811 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5a940-8927-4d17-8111-fa39bd5283b9 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:38.266746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f4b5a940-8927-4d17-8111-fa39bd5283b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:38.277523 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1413/2838] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:16:37 2022] DELETE /identity/v3/users/daa5a4286c5e421498bbf8d1c7cdd6b7 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:38.279195 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423abf03-7b39-43ee-b590-6670a26f2f8d None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:38.279606 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423abf03-7b39-43ee-b590-6670a26f2f8d None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:38.280135 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423abf03-7b39-43ee-b590-6670a26f2f8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:38.302126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b5a940-8927-4d17-8111-fa39bd5283b9 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:38.339049 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-423abf03-7b39-43ee-b590-6670a26f2f8d None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:38.342030 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-423abf03-7b39-43ee-b590-6670a26f2f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:38.464318 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476e8ac-a9b9-4c77-9df3-f22e01a327b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:38.466254 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476e8ac-a9b9-4c77-9df3-f22e01a327b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:38.467890 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476e8ac-a9b9-4c77-9df3-f22e01a327b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:38.468088 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b476e8ac-a9b9-4c77-9df3-f22e01a327b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:38.469833 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476e8ac-a9b9-4c77-9df3-f22e01a327b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:38.474575 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b476e8ac-a9b9-4c77-9df3-f22e01a327b0 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:38.507123 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1426/2839] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:38.511377 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476e8ac-a9b9-4c77-9df3-f22e01a327b0 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:38.514777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1414/2840] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) May 25 16:16:38.529752 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423abf03-7b39-43ee-b590-6670a26f2f8d None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:38.534893 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-423abf03-7b39-43ee-b590-6670a26f2f8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:38.553717 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b476e8ac-a9b9-4c77-9df3-f22e01a327b0 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:38.577093 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058b083-1147-433e-9898-ad59043a12dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:38.578140 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058b083-1147-433e-9898-ad59043a12dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:38.580159 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058b083-1147-433e-9898-ad59043a12dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:38.580442 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f058b083-1147-433e-9898-ad59043a12dc admin admin] PATH_INFO: `/v3/projects/125dc8f9e58b436a9103f12b6968a2cd/users/92a57e680b63468c8ba0c0f976434168/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:38.618346 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058b083-1147-433e-9898-ad59043a12dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=125dc8f9e58b436a9103f12b6968a2cd, user_id=92a57e680b63468c8ba0c0f976434168, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:38.621300 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f058b083-1147-433e-9898-ad59043a12dc admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:38.628209 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:38.651393 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1427/2841] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:38 2022] PUT /identity/v3/projects/125dc8f9e58b436a9103f12b6968a2cd/users/92a57e680b63468c8ba0c0f976434168/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:38.655777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058b083-1147-433e-9898-ad59043a12dc admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:38.692057 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423abf03-7b39-43ee-b590-6670a26f2f8d None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:38.790930 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f058b083-1147-433e-9898-ad59043a12dc admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:38.802112 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2ddbb-be84-4e43-9d65-236a320432b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:38.803675 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2ddbb-be84-4e43-9d65-236a320432b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:38.804368 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2ddbb-be84-4e43-9d65-236a320432b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:38.810296 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffdc62-96c2-4868-9df3-d5bb5e14e377 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:38.811084 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffdc62-96c2-4868-9df3-d5bb5e14e377 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:38.811576 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffdc62-96c2-4868-9df3-d5bb5e14e377 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:38.811935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddffdc62-96c2-4868-9df3-d5bb5e14e377 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:38.813203 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30e2ddbb-be84-4e43-9d65-236a320432b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:38.814121 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2ddbb-be84-4e43-9d65-236a320432b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:38.815876 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30e2ddbb-be84-4e43-9d65-236a320432b7 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:38.845173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1428/2842] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:38 2022] GET /identity/v3/roles => generated 1420 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:16:38.846317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffdc62-96c2-4868-9df3-d5bb5e14e377 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:38.847846 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddffdc62-96c2-4868-9df3-d5bb5e14e377 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:38.852869 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2ddbb-be84-4e43-9d65-236a320432b7 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:38.875410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30e2ddbb-be84-4e43-9d65-236a320432b7 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:38.876691 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1415/2843] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:38.889407 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44e7f9ac-6658-4506-8399-b47c2dca962b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:38.890347 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7f9ac-6658-4506-8399-b47c2dca962b admin admin] REQUEST_METHOD: `PUT` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:38.890641 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7f9ac-6658-4506-8399-b47c2dca962b admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:38.890923 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44e7f9ac-6658-4506-8399-b47c2dca962b admin admin] PATH_INFO: `/v3/projects/125dc8f9e58b436a9103f12b6968a2cd/users/92a57e680b63468c8ba0c0f976434168/roles/ca9c39f0ab2542618572a3c75e7684a1` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:38.901248 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7f9ac-6658-4506-8399-b47c2dca962b admin admin] RBAC: Authorizing `identity:create_grant(project_id=125dc8f9e58b436a9103f12b6968a2cd, user_id=92a57e680b63468c8ba0c0f976434168, role_id=ca9c39f0ab2542618572a3c75e7684a1)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:38.906793 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44e7f9ac-6658-4506-8399-b47c2dca962b admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:38.936205 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1429/2844] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:38 2022] PUT /identity/v3/projects/125dc8f9e58b436a9103f12b6968a2cd/users/92a57e680b63468c8ba0c0f976434168/roles/ca9c39f0ab2542618572a3c75e7684a1 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:39.843856 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffdc62-96c2-4868-9df3-d5bb5e14e377 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:39.845951 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ddffdc62-96c2-4868-9df3-d5bb5e14e377 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:39.892166 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddffdc62-96c2-4868-9df3-d5bb5e14e377 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:40.055459 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1cfbcaf-2f90-4b6f-902f-ddfc52b4d634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:40.056695 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfbcaf-2f90-4b6f-902f-ddfc52b4d634 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:40.057055 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfbcaf-2f90-4b6f-902f-ddfc52b4d634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:40.057343 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1cfbcaf-2f90-4b6f-902f-ddfc52b4d634 admin admin] PATH_INFO: `/v3/projects/da13efd30db34a93a36b81ff8154923d` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:40.060488 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cfbcaf-2f90-4b6f-902f-ddfc52b4d634 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da13efd30db34a93a36b81ff8154923d)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:40.074521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1cfbcaf-2f90-4b6f-902f-ddfc52b4d634 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:40.183941 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c1cfbcaf-2f90-4b6f-902f-ddfc52b4d634 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da13efd30db34a93a36b81ff8154923d', 'request_id': 'req-c1cfbcaf-2f90-4b6f-902f-ddfc52b4d634'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:40.204460 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c1cfbcaf-2f90-4b6f-902f-ddfc52b4d634 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da13efd30db34a93a36b81ff8154923d', 'request_id': 'req-c1cfbcaf-2f90-4b6f-902f-ddfc52b4d634'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:40.215871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c1cfbcaf-2f90-4b6f-902f-ddfc52b4d634 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da13efd30db34a93a36b81ff8154923d', 'request_id': 'req-c1cfbcaf-2f90-4b6f-902f-ddfc52b4d634'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:40.243646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c1cfbcaf-2f90-4b6f-902f-ddfc52b4d634 admin admin] The token cache is being invalidate because project da13efd30db34a93a36b81ff8154923d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:40.245910 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c1cfbcaf-2f90-4b6f-902f-ddfc52b4d634 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:40.256589 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c1cfbcaf-2f90-4b6f-902f-ddfc52b4d634 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:40.264601 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1416/2845] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:39 2022] DELETE /identity/v3/projects/da13efd30db34a93a36b81ff8154923d => generated 0 bytes in 416 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:40.268819 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aac1bf-0f71-4ecf-9bc1-196749a88f07 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:40.269359 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aac1bf-0f71-4ecf-9bc1-196749a88f07 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:40.269447 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91aac1bf-0f71-4ecf-9bc1-196749a88f07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:40.343942 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-91aac1bf-0f71-4ecf-9bc1-196749a88f07 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:40.350543 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-91aac1bf-0f71-4ecf-9bc1-196749a88f07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:40.376896 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:40.544319 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1430/2846] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:40.553310 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8f0d0-3115-472a-bcb0-253e966d0b89 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:40.553914 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8f0d0-3115-472a-bcb0-253e966d0b89 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:40.554306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e8f0d0-3115-472a-bcb0-253e966d0b89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:40.617250 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-19e8f0d0-3115-472a-bcb0-253e966d0b89 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:40.621568 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-19e8f0d0-3115-472a-bcb0-253e966d0b89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:40.652241 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1417/2847] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:40.653327 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aac1bf-0f71-4ecf-9bc1-196749a88f07 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:40.666414 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91aac1bf-0f71-4ecf-9bc1-196749a88f07 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:40.681800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c8cc7-51f7-444e-913e-a3fd8ad57965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:40.692636 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c8cc7-51f7-444e-913e-a3fd8ad57965 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:40.693049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c8cc7-51f7-444e-913e-a3fd8ad57965 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:40.693361 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b45c8cc7-51f7-444e-913e-a3fd8ad57965 admin admin] PATH_INFO: `/v3/domains` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:40.694090 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c8cc7-51f7-444e-913e-a3fd8ad57965 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80232) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} May 25 16:16:40.698851 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c8cc7-51f7-444e-913e-a3fd8ad57965 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:40.700800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b45c8cc7-51f7-444e-913e-a3fd8ad57965 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:40.763042 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1431/2848] 10.222.0.85 () {58 vars in 1152 bytes} [Wed May 25 16:16:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) May 25 16:16:40.767754 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8f0d0-3115-472a-bcb0-253e966d0b89 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:40.785806 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e8f0d0-3115-472a-bcb0-253e966d0b89 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:40.808148 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cd925-a544-4c93-8f8b-ae2d08375a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:40.809859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cd925-a544-4c93-8f8b-ae2d08375a0f admin admin] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:40.810317 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cd925-a544-4c93-8f8b-ae2d08375a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:40.810718 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7cd925-a544-4c93-8f8b-ae2d08375a0f admin admin] PATH_INFO: `/v3/projects` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:40.812007 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cd925-a544-4c93-8f8b-ae2d08375a0f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:40.815178 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7cd925-a544-4c93-8f8b-ae2d08375a0f admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:40.833817 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:40.877140 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1418/2849] 10.222.0.85 () {60 vars in 1152 bytes} [Wed May 25 16:16:40 2022] POST /identity/v3/projects => generated 375 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) May 25 16:16:40.877775 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c8cc7-51f7-444e-913e-a3fd8ad57965 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:40.893331 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b45c8cc7-51f7-444e-913e-a3fd8ad57965 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:41.017197 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c8b79-6dab-43ed-bd4b-8bab12bab03e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:41.018827 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c8b79-6dab-43ed-bd4b-8bab12bab03e admin admin] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:41.019222 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c8b79-6dab-43ed-bd4b-8bab12bab03e admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:41.019616 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-221c8b79-6dab-43ed-bd4b-8bab12bab03e admin admin] PATH_INFO: `/v3/users` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:41.020497 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c8b79-6dab-43ed-bd4b-8bab12bab03e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:41.023229 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-221c8b79-6dab-43ed-bd4b-8bab12bab03e admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:41.076000 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1432/2850] 10.222.0.85 () {60 vars in 1143 bytes} [Wed May 25 16:16:40 2022] POST /identity/v3/users => generated 288 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) May 25 16:16:41.080290 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cd925-a544-4c93-8f8b-ae2d08375a0f admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:41.108065 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7cd925-a544-4c93-8f8b-ae2d08375a0f admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:41.134318 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75219d0f-eeb0-489e-9eb3-a83e4eb5ff7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:41.134318 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75219d0f-eeb0-489e-9eb3-a83e4eb5ff7c admin admin] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:41.134318 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75219d0f-eeb0-489e-9eb3-a83e4eb5ff7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:41.134318 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75219d0f-eeb0-489e-9eb3-a83e4eb5ff7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:41.134895 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75219d0f-eeb0-489e-9eb3-a83e4eb5ff7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:41.136921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75219d0f-eeb0-489e-9eb3-a83e4eb5ff7c admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:41.157571 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1419/2851] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) May 25 16:16:41.161592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c8b79-6dab-43ed-bd4b-8bab12bab03e admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:41.177257 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-221c8b79-6dab-43ed-bd4b-8bab12bab03e admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:41.191007 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d548af3-a655-4edd-a118-10988f9ed9e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:41.192785 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d548af3-a655-4edd-a118-10988f9ed9e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:41.193173 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d548af3-a655-4edd-a118-10988f9ed9e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:41.193405 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d548af3-a655-4edd-a118-10988f9ed9e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:41.218434 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d548af3-a655-4edd-a118-10988f9ed9e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:41.221876 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d548af3-a655-4edd-a118-10988f9ed9e1 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:41.246699 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1433/2852] 10.222.0.85 () {58 vars in 1121 bytes} [Wed May 25 16:16:41 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) May 25 16:16:41.250780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75219d0f-eeb0-489e-9eb3-a83e4eb5ff7c admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:41.265536 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75219d0f-eeb0-489e-9eb3-a83e4eb5ff7c admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:41.271784 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494ef3b-2316-48ba-894f-f751480a15ae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:41.273023 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494ef3b-2316-48ba-894f-f751480a15ae admin admin] REQUEST_METHOD: `PUT` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:41.273388 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494ef3b-2316-48ba-894f-f751480a15ae admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:41.273739 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3494ef3b-2316-48ba-894f-f751480a15ae admin admin] PATH_INFO: `/v3/projects/85a09d3b9a1741c29dd8dfe28259a95c/users/5c7abfc391414632b0d1ca4e682d9995/roles/9c81542451e54c0ea22a20c026be6be7` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:41.305107 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494ef3b-2316-48ba-894f-f751480a15ae admin admin] RBAC: Authorizing `identity:create_grant(project_id=85a09d3b9a1741c29dd8dfe28259a95c, user_id=5c7abfc391414632b0d1ca4e682d9995, role_id=9c81542451e54c0ea22a20c026be6be7)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:41.308018 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3494ef3b-2316-48ba-894f-f751480a15ae admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:41.349234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1420/2853] 10.222.0.85 () {60 vars in 1482 bytes} [Wed May 25 16:16:41 2022] PUT /identity/v3/projects/85a09d3b9a1741c29dd8dfe28259a95c/users/5c7abfc391414632b0d1ca4e682d9995/roles/9c81542451e54c0ea22a20c026be6be7 => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:41.417609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330af55d-55df-44cf-9c6d-7d95adcdd231 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:41.418726 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330af55d-55df-44cf-9c6d-7d95adcdd231 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:41.419456 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330af55d-55df-44cf-9c6d-7d95adcdd231 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:41.456969 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-330af55d-55df-44cf-9c6d-7d95adcdd231 None None] MFA Rules not processed for user `5c7abfc391414632b0d1ca4e682d9995`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:41.460038 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-330af55d-55df-44cf-9c6d-7d95adcdd231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:41.586534 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1434/2854] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:41.600689 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494ef3b-2316-48ba-894f-f751480a15ae admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:41.613747 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-3494ef3b-2316-48ba-894f-f751480a15ae admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:41.767804 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3494ef3b-2316-48ba-894f-f751480a15ae admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:41.779292 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59760a26-3a49-44dd-9c0d-00740dec356a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f81aecdc87d84501ba94b208405b51dd', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:41.780017 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59760a26-3a49-44dd-9c0d-00740dec356a service cinder] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:41.780304 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59760a26-3a49-44dd-9c0d-00740dec356a service cinder] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:41.780594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59760a26-3a49-44dd-9c0d-00740dec356a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:41.801084 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59760a26-3a49-44dd-9c0d-00740dec356a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:41.802688 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59760a26-3a49-44dd-9c0d-00740dec356a service cinder] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:41.863732 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1421/2855] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:41 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 254 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:48.853080 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69691e-2af9-4142-8654-03b4352c0a8b None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:48.854490 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69691e-2af9-4142-8654-03b4352c0a8b None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:48.854987 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f69691e-2af9-4142-8654-03b4352c0a8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:48.882479 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:48.896323 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-8f69691e-2af9-4142-8654-03b4352c0a8b None None] MFA Rules not processed for user `92a57e680b63468c8ba0c0f976434168`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:48.897684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-8f69691e-2af9-4142-8654-03b4352c0a8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:49.000601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1435/2856] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:48 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:49.018281 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59760a26-3a49-44dd-9c0d-00740dec356a service cinder] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:49.020463 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-59760a26-3a49-44dd-9c0d-00740dec356a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:49.110280 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59760a26-3a49-44dd-9c0d-00740dec356a service cinder] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:49.116340 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6ea3776-525f-4f3e-b4d8-043182450b2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:49.117028 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea3776-525f-4f3e-b4d8-043182450b2e service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:49.117319 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea3776-525f-4f3e-b4d8-043182450b2e service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:49.117741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ea3776-525f-4f3e-b4d8-043182450b2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:49.119498 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a6ea3776-525f-4f3e-b4d8-043182450b2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:49.211527 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:49.214176 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea3776-525f-4f3e-b4d8-043182450b2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:49.217391 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6ea3776-525f-4f3e-b4d8-043182450b2e service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:49.230785 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1422/2857] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:49 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:54.394928 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69691e-2af9-4142-8654-03b4352c0a8b None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:54.465857 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f69691e-2af9-4142-8654-03b4352c0a8b None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:54.473094 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d300d-b605-483c-8baa-5cb7cf157022 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:54.474633 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d300d-b605-483c-8baa-5cb7cf157022 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:54.474926 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d300d-b605-483c-8baa-5cb7cf157022 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:54.475216 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3d300d-b605-483c-8baa-5cb7cf157022 admin admin] PATH_INFO: `/v3/users/5c7abfc391414632b0d1ca4e682d9995` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:54.476529 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d300d-b605-483c-8baa-5cb7cf157022 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5c7abfc391414632b0d1ca4e682d9995)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:54.478473 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3d300d-b605-483c-8baa-5cb7cf157022 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:54.536174 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cf3d300d-b605-483c-8baa-5cb7cf157022 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5c7abfc391414632b0d1ca4e682d9995'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:54.537261 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cf3d300d-b605-483c-8baa-5cb7cf157022 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5c7abfc391414632b0d1ca4e682d9995'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:54.550792 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cf3d300d-b605-483c-8baa-5cb7cf157022 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5c7abfc391414632b0d1ca4e682d9995'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:54.559491 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cf3d300d-b605-483c-8baa-5cb7cf157022 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5c7abfc391414632b0d1ca4e682d9995'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:54.560235 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-cf3d300d-b605-483c-8baa-5cb7cf157022 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5c7abfc391414632b0d1ca4e682d9995'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:54.577935 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1436/2858] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:16:54 2022] DELETE /identity/v3/users/5c7abfc391414632b0d1ca4e682d9995 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:16:54.582560 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8cd68-5197-4e25-8611-f20a3cc33bf6 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:54.583557 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8cd68-5197-4e25-8611-f20a3cc33bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:54.583898 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc8cd68-5197-4e25-8611-f20a3cc33bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:54.608908 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-ffc8cd68-5197-4e25-8611-f20a3cc33bf6 None None] MFA Rules not processed for user `7effefc613a44b799e8f81fc6a35b678`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:16:54.610202 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ffc8cd68-5197-4e25-8611-f20a3cc33bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:54.689409 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1423/2859] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:16:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:16:54.709052 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d300d-b605-483c-8baa-5cb7cf157022 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:54.711179 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-cf3d300d-b605-483c-8baa-5cb7cf157022 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:54.738124 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:16:54.803353 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3d300d-b605-483c-8baa-5cb7cf157022 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:54.808609 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b58334aa-0642-4789-afb9-9a4f1832e9ab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:54.809574 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58334aa-0642-4789-afb9-9a4f1832e9ab service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:54.809983 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58334aa-0642-4789-afb9-9a4f1832e9ab service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:54.810329 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b58334aa-0642-4789-afb9-9a4f1832e9ab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:54.820970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58334aa-0642-4789-afb9-9a4f1832e9ab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:54.822592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b58334aa-0642-4789-afb9-9a4f1832e9ab service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:54.832968 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1437/2860] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:16:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:16:55.409212 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8cd68-5197-4e25-8611-f20a3cc33bf6 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:16:55.410864 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-ffc8cd68-5197-4e25-8611-f20a3cc33bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:16:55.428625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc8cd68-5197-4e25-8611-f20a3cc33bf6 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:16:55.435585 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1c1da-721c-4e62-971d-f079de945f05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:16:55.436503 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1c1da-721c-4e62-971d-f079de945f05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:16:55.436796 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1c1da-721c-4e62-971d-f079de945f05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:16:55.437130 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d1c1da-721c-4e62-971d-f079de945f05 admin admin] PATH_INFO: `/v3/projects/85a09d3b9a1741c29dd8dfe28259a95c` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:16:55.438378 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1c1da-721c-4e62-971d-f079de945f05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=85a09d3b9a1741c29dd8dfe28259a95c)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:16:55.440520 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d1c1da-721c-4e62-971d-f079de945f05 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:16:55.490066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c4d1c1da-721c-4e62-971d-f079de945f05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85a09d3b9a1741c29dd8dfe28259a95c', 'request_id': 'req-c4d1c1da-721c-4e62-971d-f079de945f05'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:55.498197 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c4d1c1da-721c-4e62-971d-f079de945f05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85a09d3b9a1741c29dd8dfe28259a95c', 'request_id': 'req-c4d1c1da-721c-4e62-971d-f079de945f05'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:55.506882 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c4d1c1da-721c-4e62-971d-f079de945f05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85a09d3b9a1741c29dd8dfe28259a95c', 'request_id': 'req-c4d1c1da-721c-4e62-971d-f079de945f05'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:55.519933 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c4d1c1da-721c-4e62-971d-f079de945f05 admin admin] The token cache is being invalidate because project 85a09d3b9a1741c29dd8dfe28259a95c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:16:55.520207 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c4d1c1da-721c-4e62-971d-f079de945f05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:55.520879 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-c4d1c1da-721c-4e62-971d-f079de945f05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:16:55.522468 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1424/2861] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:16:55 2022] DELETE /identity/v3/projects/85a09d3b9a1741c29dd8dfe28259a95c => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:17:04.828237 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=20 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:05.517043 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=56,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:05.994987 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0086d-b2dc-4ce3-807d-d52a90bf58f6 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:17:05.995608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0086d-b2dc-4ce3-807d-d52a90bf58f6 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:17:05.995608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db0086d-b2dc-4ce3-807d-d52a90bf58f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:17:06.014973 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-3db0086d-b2dc-4ce3-807d-d52a90bf58f6 None None] MFA Rules not processed for user `67b7bca2e2ef4dbf89554ba2f9bf93e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:17:06.016234 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3db0086d-b2dc-4ce3-807d-d52a90bf58f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:17:06.075856 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1438/2862] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:17:05 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:17:07.293367 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1c1da-721c-4e62-971d-f079de945f05 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:17:07.295178 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c4d1c1da-721c-4e62-971d-f079de945f05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:17:07.361131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d1c1da-721c-4e62-971d-f079de945f05 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:17:07.365504 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa1e68-892e-4a27-8327-e245f1c18064 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:17:07.366234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa1e68-892e-4a27-8327-e245f1c18064 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:17:07.366532 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa1e68-892e-4a27-8327-e245f1c18064 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:17:07.366823 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9aa1e68-892e-4a27-8327-e245f1c18064 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:17:07.368828 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-e9aa1e68-892e-4a27-8327-e245f1c18064 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:17:07.438756 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa1e68-892e-4a27-8327-e245f1c18064 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:17:07.440389 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9aa1e68-892e-4a27-8327-e245f1c18064 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:17:07.451341 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1425/2863] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:17:07 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:17:16.069172 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=25 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:17.447268 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=50 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:17:24.038360 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0086d-b2dc-4ce3-807d-d52a90bf58f6 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:17:24.039928 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-3db0086d-b2dc-4ce3-807d-d52a90bf58f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:17:24.053299 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db0086d-b2dc-4ce3-807d-d52a90bf58f6 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:17:24.057683 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2d9fe-4007-4541-abfb-911acfbe065c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:17:24.058340 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2d9fe-4007-4541-abfb-911acfbe065c service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:17:24.058633 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2d9fe-4007-4541-abfb-911acfbe065c service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:17:24.058907 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2d9fe-4007-4541-abfb-911acfbe065c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:17:24.060914 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-4aa2d9fe-4007-4541-abfb-911acfbe065c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:17:24.130109 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2d9fe-4007-4541-abfb-911acfbe065c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:17:24.132592 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2d9fe-4007-4541-abfb-911acfbe065c service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:17:24.147291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1439/2864] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:17:24 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:17:34.141290 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=30 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:06.460001 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa1e68-892e-4a27-8327-e245f1c18064 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:06.469454 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9aa1e68-892e-4a27-8327-e245f1c18064 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:06.473792 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2686ec0-02fc-4f09-8272-e81847032159 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:06.474453 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2686ec0-02fc-4f09-8272-e81847032159 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:06.474805 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2686ec0-02fc-4f09-8272-e81847032159 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:06.475089 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2686ec0-02fc-4f09-8272-e81847032159 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:06.477036 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-f2686ec0-02fc-4f09-8272-e81847032159 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:06.553082 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2686ec0-02fc-4f09-8272-e81847032159 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:06.554541 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2686ec0-02fc-4f09-8272-e81847032159 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:06.565805 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1426/2865] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:18:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:07.757727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2d9fe-4007-4541-abfb-911acfbe065c service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:07.769079 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2d9fe-4007-4541-abfb-911acfbe065c service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:07.774390 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f79bae-a4ea-4dba-a78a-65559c311865 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:07.775120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f79bae-a4ea-4dba-a78a-65559c311865 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:07.775484 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f79bae-a4ea-4dba-a78a-65559c311865 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:07.775891 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f79bae-a4ea-4dba-a78a-65559c311865 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:07.778334 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-43f79bae-a4ea-4dba-a78a-65559c311865 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:07.794016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f79bae-a4ea-4dba-a78a-65559c311865 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:07.795462 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f79bae-a4ea-4dba-a78a-65559c311865 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:07.806823 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1440/2866] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:18:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 51 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:16.561065 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=34 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:16.616729 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2686ec0-02fc-4f09-8272-e81847032159 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:16.626181 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2686ec0-02fc-4f09-8272-e81847032159 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:16.630949 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5fb7737-bb1c-49bd-bb60-40677d9bd376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:16.631605 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb7737-bb1c-49bd-bb60-40677d9bd376 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:16.631937 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb7737-bb1c-49bd-bb60-40677d9bd376 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:16.632223 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5fb7737-bb1c-49bd-bb60-40677d9bd376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:16.634615 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b5fb7737-bb1c-49bd-bb60-40677d9bd376 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:16.645521 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb7737-bb1c-49bd-bb60-40677d9bd376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:16.647795 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5fb7737-bb1c-49bd-bb60-40677d9bd376 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:16.660660 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1427/2867] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:18:16 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:17.802160 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=6 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:20.012416 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f79bae-a4ea-4dba-a78a-65559c311865 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:20.021981 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f79bae-a4ea-4dba-a78a-65559c311865 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:20.025992 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f2d892a-00a0-437f-a798-f08aeb759b67 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:20.026639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d892a-00a0-437f-a798-f08aeb759b67 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:20.026930 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d892a-00a0-437f-a798-f08aeb759b67 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:20.027206 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f2d892a-00a0-437f-a798-f08aeb759b67 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:20.029186 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-9f2d892a-00a0-437f-a798-f08aeb759b67 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:20.039573 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d892a-00a0-437f-a798-f08aeb759b67 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:20.041072 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f2d892a-00a0-437f-a798-f08aeb759b67 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:20.049722 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1441/2868] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:18:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:20.155224 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be245e-a9d5-4e89-9bb2-778ba9d69d79 None None] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:20.155902 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be245e-a9d5-4e89-9bb2-778ba9d69d79 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:20.155902 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be245e-a9d5-4e89-9bb2-778ba9d69d79 None None] PATH_INFO: `/` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:20.157213 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1428/2869] 10.222.0.85 () {54 vars in 921 bytes} [Wed May 25 16:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:18:20.163676 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1994a-1b81-418b-9dce-a52efa86c51a None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:20.164280 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1994a-1b81-418b-9dce-a52efa86c51a None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:20.164540 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b1994a-1b81-418b-9dce-a52efa86c51a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:20.203925 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f6b1994a-1b81-418b-9dce-a52efa86c51a None None] MFA Rules not processed for user `413b3d65427b41dc99bc7b0a94fb1dc8`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:18:20.205144 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f6b1994a-1b81-418b-9dce-a52efa86c51a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:20.212521 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1442/2870] 10.222.0.85 () {58 vars in 1018 bytes} [Wed May 25 16:18:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:18:20.225535 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be245e-a9d5-4e89-9bb2-778ba9d69d79 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:20.237867 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be245e-a9d5-4e89-9bb2-778ba9d69d79 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:20.243921 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9be340-827d-441c-aaba-1f22f5fdd4da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:20.244625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9be340-827d-441c-aaba-1f22f5fdd4da service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:20.245039 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9be340-827d-441c-aaba-1f22f5fdd4da service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:20.245343 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9be340-827d-441c-aaba-1f22f5fdd4da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:20.253259 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9be340-827d-441c-aaba-1f22f5fdd4da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:20.254874 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9be340-827d-441c-aaba-1f22f5fdd4da service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:20.264179 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1429/2871] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:18:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:20.354068 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ba562-f2ef-4cc9-8ea5-14ed67cd7b07 None None] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:20.354361 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ba562-f2ef-4cc9-8ea5-14ed67cd7b07 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:20.354639 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4ba562-f2ef-4cc9-8ea5-14ed67cd7b07 None None] PATH_INFO: `/` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:20.356151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1443/2872] 10.222.0.85 () {54 vars in 921 bytes} [Wed May 25 16:18:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) May 25 16:18:20.361234 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3824a7-f109-4c38-87d0-4122b7f06352 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:20.361530 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3824a7-f109-4c38-87d0-4122b7f06352 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:20.361816 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3824a7-f109-4c38-87d0-4122b7f06352 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:20.381625 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-bb3824a7-f109-4c38-87d0-4122b7f06352 None None] MFA Rules not processed for user `4751abc9f67947b3ad17ba2b78ce2c12`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:18:20.382826 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bb3824a7-f109-4c38-87d0-4122b7f06352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:20.392591 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1430/2873] 10.222.0.85 () {58 vars in 1018 bytes} [Wed May 25 16:18:20 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:18:20.402951 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ba562-f2ef-4cc9-8ea5-14ed67cd7b07 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:20.415857 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4ba562-f2ef-4cc9-8ea5-14ed67cd7b07 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:20.419879 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4201428-7839-4ede-8266-8e778821fc15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:20.420518 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4201428-7839-4ede-8266-8e778821fc15 service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:20.420808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4201428-7839-4ede-8266-8e778821fc15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:20.421087 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4201428-7839-4ede-8266-8e778821fc15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:20.431785 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4201428-7839-4ede-8266-8e778821fc15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:20.433255 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4201428-7839-4ede-8266-8e778821fc15 service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:20.444192 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1444/2874] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:18:20 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:20.529266 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3824a7-f109-4c38-87d0-4122b7f06352 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:20.530763 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-bb3824a7-f109-4c38-87d0-4122b7f06352 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:20.606439 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3824a7-f109-4c38-87d0-4122b7f06352 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:20.611982 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b532e6-762a-4085-a846-69c7825b7c53 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b738f1b8c64590955a640c461b4ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:20.612665 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b532e6-762a-4085-a846-69c7825b7c53 service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:20.612950 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b532e6-762a-4085-a846-69c7825b7c53 service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:20.613222 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b532e6-762a-4085-a846-69c7825b7c53 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:20.615331 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-12b532e6-762a-4085-a846-69c7825b7c53 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:20.627352 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b532e6-762a-4085-a846-69c7825b7c53 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:20.629121 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b532e6-762a-4085-a846-69c7825b7c53 service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:20.639021 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1431/2875] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:18:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:30.144549 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4201428-7839-4ede-8266-8e778821fc15 service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:30.146527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f4201428-7839-4ede-8266-8e778821fc15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:30.162909 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4201428-7839-4ede-8266-8e778821fc15 service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:30.169421 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5152b-281f-48c3-814f-5d2ee58d0eb3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:30.170118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5152b-281f-48c3-814f-5d2ee58d0eb3 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:30.170469 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5152b-281f-48c3-814f-5d2ee58d0eb3 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:30.170735 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f5152b-281f-48c3-814f-5d2ee58d0eb3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:30.172608 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-61f5152b-281f-48c3-814f-5d2ee58d0eb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:30.185839 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5152b-281f-48c3-814f-5d2ee58d0eb3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:30.187560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f5152b-281f-48c3-814f-5d2ee58d0eb3 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:30.198678 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1445/2876] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:18:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:30.634859 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=47 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:32.005920 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1184e9-d899-4bff-95fd-aea8ead830e7 None None] REQUEST_METHOD: `POST` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:32.006506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1184e9-d899-4bff-95fd-aea8ead830e7 None None] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:32.006506 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1184e9-d899-4bff-95fd-aea8ead830e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:32.025459 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.auth.core [None req-9c1184e9-d899-4bff-95fd-aea8ead830e7 None None] MFA Rules not processed for user `066de1f11f724eee9df0377beb58d92c`. Rule list: `[]` (Enabled: `True`). {{(pid=80233) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:18:32.026659 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-9c1184e9-d899-4bff-95fd-aea8ead830e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:32.084083 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1432/2877] 10.222.0.85 () {58 vars in 967 bytes} [Wed May 25 16:18:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:18:32.125178 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5152b-281f-48c3-814f-5d2ee58d0eb3 service nova] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:32.126777 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-61f5152b-281f-48c3-814f-5d2ee58d0eb3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:32.197937 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f5152b-281f-48c3-814f-5d2ee58d0eb3 service nova] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:32.203014 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75d732-784b-4273-bab7-d49fbc22d1c6 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '77fda27641c84fccb68a5cdaf4e8dc4b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:32.203780 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75d732-784b-4273-bab7-d49fbc22d1c6 service swift] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:32.203932 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75d732-784b-4273-bab7-d49fbc22d1c6 service swift] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:32.204394 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c75d732-784b-4273-bab7-d49fbc22d1c6 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:32.213000 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c75d732-784b-4273-bab7-d49fbc22d1c6 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:32.214581 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c75d732-784b-4273-bab7-d49fbc22d1c6 service swift] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:32.222697 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1446/2878] 10.222.0.85 () {60 vars in 1384 bytes} [Wed May 25 16:18:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) May 25 16:18:33.211534 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1184e9-d899-4bff-95fd-aea8ead830e7 None None] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:33.226540 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1184e9-d899-4bff-95fd-aea8ead830e7 None None] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:33.232373 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69bd18-15ec-4bf6-b1ba-61f5a27e6beb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:33.233383 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69bd18-15ec-4bf6-b1ba-61f5a27e6beb service nova] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:33.233847 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69bd18-15ec-4bf6-b1ba-61f5a27e6beb service nova] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:33.234139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a69bd18-15ec-4bf6-b1ba-61f5a27e6beb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:33.236646 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-5a69bd18-15ec-4bf6-b1ba-61f5a27e6beb service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:33.250777 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69bd18-15ec-4bf6-b1ba-61f5a27e6beb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:33.253315 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a69bd18-15ec-4bf6-b1ba-61f5a27e6beb service nova] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:33.266428 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1433/2879] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:18:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:18:39.017116 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75d732-784b-4273-bab7-d49fbc22d1c6 service swift] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:39.018744 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0c75d732-784b-4273-bab7-d49fbc22d1c6 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:39.087748 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c75d732-784b-4273-bab7-d49fbc22d1c6 service swift] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:39.092460 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1027f-0e20-43db-97a4-f3425454e961 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:39.093361 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1027f-0e20-43db-97a4-f3425454e961 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:39.093770 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1027f-0e20-43db-97a4-f3425454e961 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:39.094056 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1e1027f-0e20-43db-97a4-f3425454e961 admin admin] PATH_INFO: `/v3/users/920a00aa201143f6ae4799aff51c6dae` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:39.095255 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1027f-0e20-43db-97a4-f3425454e961 admin admin] RBAC: Authorizing `identity:delete_user(user_id=920a00aa201143f6ae4799aff51c6dae)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:39.097217 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1e1027f-0e20-43db-97a4-f3425454e961 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:39.133336 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:39.146719 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b1e1027f-0e20-43db-97a4-f3425454e961 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '920a00aa201143f6ae4799aff51c6dae'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:39.147379 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b1e1027f-0e20-43db-97a4-f3425454e961 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '920a00aa201143f6ae4799aff51c6dae'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:39.159763 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b1e1027f-0e20-43db-97a4-f3425454e961 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '920a00aa201143f6ae4799aff51c6dae'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:39.168105 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b1e1027f-0e20-43db-97a4-f3425454e961 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '920a00aa201143f6ae4799aff51c6dae'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:39.168732 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-b1e1027f-0e20-43db-97a4-f3425454e961 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '920a00aa201143f6ae4799aff51c6dae'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:39.184242 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1447/2880] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:18:39 2022] DELETE /identity/v3/users/920a00aa201143f6ae4799aff51c6dae => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:18:39.364231 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69bd18-15ec-4bf6-b1ba-61f5a27e6beb service nova] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:39.376766 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a69bd18-15ec-4bf6-b1ba-61f5a27e6beb service nova] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:39.442386 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca3cc5-ba25-4656-a0e0-6394a6222f03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:39.443299 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca3cc5-ba25-4656-a0e0-6394a6222f03 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:39.443592 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca3cc5-ba25-4656-a0e0-6394a6222f03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:39.443905 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ca3cc5-ba25-4656-a0e0-6394a6222f03 admin admin] PATH_INFO: `/v3/projects/e2fbd40b4ea447319a8b90656548da9e` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:39.445345 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca3cc5-ba25-4656-a0e0-6394a6222f03 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2fbd40b4ea447319a8b90656548da9e)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:39.447306 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ca3cc5-ba25-4656-a0e0-6394a6222f03 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:39.498528 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-82ca3cc5-ba25-4656-a0e0-6394a6222f03 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2fbd40b4ea447319a8b90656548da9e', 'request_id': 'req-82ca3cc5-ba25-4656-a0e0-6394a6222f03'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:39.508167 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-82ca3cc5-ba25-4656-a0e0-6394a6222f03 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2fbd40b4ea447319a8b90656548da9e', 'request_id': 'req-82ca3cc5-ba25-4656-a0e0-6394a6222f03'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:39.515128 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-82ca3cc5-ba25-4656-a0e0-6394a6222f03 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2fbd40b4ea447319a8b90656548da9e', 'request_id': 'req-82ca3cc5-ba25-4656-a0e0-6394a6222f03'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:39.526605 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-82ca3cc5-ba25-4656-a0e0-6394a6222f03 admin admin] The token cache is being invalidate because project e2fbd40b4ea447319a8b90656548da9e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:18:39.526949 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-82ca3cc5-ba25-4656-a0e0-6394a6222f03 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:39.527632 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-82ca3cc5-ba25-4656-a0e0-6394a6222f03 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:39.529271 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1434/2881] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:18:39 2022] DELETE /identity/v3/projects/e2fbd40b4ea447319a8b90656548da9e => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:18:39.533613 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1027f-0e20-43db-97a4-f3425454e961 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:39.535127 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b1e1027f-0e20-43db-97a4-f3425454e961 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:39.548842 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1e1027f-0e20-43db-97a4-f3425454e961 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:39.608555 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c84138b-f15b-4e2b-abc5-2b2dabe51e3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:39.609496 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84138b-f15b-4e2b-abc5-2b2dabe51e3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:39.609782 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84138b-f15b-4e2b-abc5-2b2dabe51e3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:39.610063 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c84138b-f15b-4e2b-abc5-2b2dabe51e3d admin admin] PATH_INFO: `/v3/users/1b008d0814014b979f86ddfebc1e4e24` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:39.611291 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84138b-f15b-4e2b-abc5-2b2dabe51e3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=1b008d0814014b979f86ddfebc1e4e24)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:39.613285 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c84138b-f15b-4e2b-abc5-2b2dabe51e3d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:39.664082 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9c84138b-f15b-4e2b-abc5-2b2dabe51e3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1b008d0814014b979f86ddfebc1e4e24'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:39.664785 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9c84138b-f15b-4e2b-abc5-2b2dabe51e3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1b008d0814014b979f86ddfebc1e4e24'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:39.676569 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9c84138b-f15b-4e2b-abc5-2b2dabe51e3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1b008d0814014b979f86ddfebc1e4e24'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:39.686108 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9c84138b-f15b-4e2b-abc5-2b2dabe51e3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1b008d0814014b979f86ddfebc1e4e24'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:39.686829 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-9c84138b-f15b-4e2b-abc5-2b2dabe51e3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1b008d0814014b979f86ddfebc1e4e24'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:39.699372 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1448/2882] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:18:39 2022] DELETE /identity/v3/users/1b008d0814014b979f86ddfebc1e4e24 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:18:39.874273 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca3cc5-ba25-4656-a0e0-6394a6222f03 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:18:39.875958 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-82ca3cc5-ba25-4656-a0e0-6394a6222f03 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:18:39.890459 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ca3cc5-ba25-4656-a0e0-6394a6222f03 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:18:39.911465 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:39.955499 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbdfaa-e743-411a-9e3c-9f7313df91b5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:18:39.956547 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbdfaa-e743-411a-9e3c-9f7313df91b5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:18:39.956844 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbdfaa-e743-411a-9e3c-9f7313df91b5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:18:39.957131 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acbdfaa-e743-411a-9e3c-9f7313df91b5 admin admin] PATH_INFO: `/v3/projects/22e9ac6dbb5149379466be8e688c58bf` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:18:39.958428 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbdfaa-e743-411a-9e3c-9f7313df91b5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=22e9ac6dbb5149379466be8e688c58bf)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:18:39.960481 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1acbdfaa-e743-411a-9e3c-9f7313df91b5 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:18:40.010227 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1acbdfaa-e743-411a-9e3c-9f7313df91b5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '22e9ac6dbb5149379466be8e688c58bf', 'request_id': 'req-1acbdfaa-e743-411a-9e3c-9f7313df91b5'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:40.018720 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1acbdfaa-e743-411a-9e3c-9f7313df91b5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '22e9ac6dbb5149379466be8e688c58bf', 'request_id': 'req-1acbdfaa-e743-411a-9e3c-9f7313df91b5'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:40.025672 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1acbdfaa-e743-411a-9e3c-9f7313df91b5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '22e9ac6dbb5149379466be8e688c58bf', 'request_id': 'req-1acbdfaa-e743-411a-9e3c-9f7313df91b5'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:40.037106 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1acbdfaa-e743-411a-9e3c-9f7313df91b5 admin admin] The token cache is being invalidate because project 22e9ac6dbb5149379466be8e688c58bf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80233) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:18:40.037732 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1acbdfaa-e743-411a-9e3c-9f7313df91b5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:40.038393 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-1acbdfaa-e743-411a-9e3c-9f7313df91b5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:18:40.039976 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1435/2883] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:18:39 2022] DELETE /identity/v3/projects/22e9ac6dbb5149379466be8e688c58bf => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:18:49.696107 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=70,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:18:50.034447 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=41,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:27.212105 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1e72d-6871-4e2d-a307-bea9b46345e1 None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:19:27.212620 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1e72d-6871-4e2d-a307-bea9b46345e1 None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:19:27.212818 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4d1e72d-6871-4e2d-a307-bea9b46345e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:19:27.231374 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-b4d1e72d-6871-4e2d-a307-bea9b46345e1 None None] MFA Rules not processed for user `67b7bca2e2ef4dbf89554ba2f9bf93e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:19:27.232626 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b4d1e72d-6871-4e2d-a307-bea9b46345e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:19:27.287908 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1449/2884] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:19:27 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:19:37.280895 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=25 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:50.459728 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbdfaa-e743-411a-9e3c-9f7313df91b5 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:19:50.461389 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-1acbdfaa-e743-411a-9e3c-9f7313df91b5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:19:50.587451 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acbdfaa-e743-411a-9e3c-9f7313df91b5 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:19:50.591916 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051bece-b0bd-4c74-98d7-36eac8390d6f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:19:50.592579 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051bece-b0bd-4c74-98d7-36eac8390d6f service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:19:50.592871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051bece-b0bd-4c74-98d7-36eac8390d6f service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:19:50.593132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a051bece-b0bd-4c74-98d7-36eac8390d6f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:19:50.595286 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a051bece-b0bd-4c74-98d7-36eac8390d6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:19:50.709406 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051bece-b0bd-4c74-98d7-36eac8390d6f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:19:50.710865 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a051bece-b0bd-4c74-98d7-36eac8390d6f service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:19:50.731272 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1436/2885] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:19:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:19:53.863214 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1e72d-6871-4e2d-a307-bea9b46345e1 None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:19:53.864835 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-b4d1e72d-6871-4e2d-a307-bea9b46345e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:19:53.985174 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4d1e72d-6871-4e2d-a307-bea9b46345e1 None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:19:53.990617 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42e406-8de8-49af-a3ed-3b435353476a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b738f1b8c64590955a640c461b4ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:19:53.991348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42e406-8de8-49af-a3ed-3b435353476a service placement] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:19:53.991703 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42e406-8de8-49af-a3ed-3b435353476a service placement] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:19:53.992129 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac42e406-8de8-49af-a3ed-3b435353476a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:19:53.993970 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ac42e406-8de8-49af-a3ed-3b435353476a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:19:54.012510 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42e406-8de8-49af-a3ed-3b435353476a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:19:54.014126 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac42e406-8de8-49af-a3ed-3b435353476a service placement] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:19:54.024412 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1450/2886] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:19:53 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:19:57.165696 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051bece-b0bd-4c74-98d7-36eac8390d6f service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:19:57.167333 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-a051bece-b0bd-4c74-98d7-36eac8390d6f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:19:57.181916 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a051bece-b0bd-4c74-98d7-36eac8390d6f service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:19:57.187368 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b822d-f384-46d1-9c9d-0f37b11bc3fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:19:57.188330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b822d-f384-46d1-9c9d-0f37b11bc3fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:19:57.188617 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b822d-f384-46d1-9c9d-0f37b11bc3fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:19:57.188871 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d15b822d-f384-46d1-9c9d-0f37b11bc3fe admin admin] PATH_INFO: `/v3/users/abd62b4b16da4dcd8e0bdbcb6d9db3a8` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:19:57.190235 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b822d-f384-46d1-9c9d-0f37b11bc3fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=abd62b4b16da4dcd8e0bdbcb6d9db3a8)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:19:57.192329 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d15b822d-f384-46d1-9c9d-0f37b11bc3fe admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:19:57.198662 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=100 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:57.243364 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d15b822d-f384-46d1-9c9d-0f37b11bc3fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'abd62b4b16da4dcd8e0bdbcb6d9db3a8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:57.256475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d15b822d-f384-46d1-9c9d-0f37b11bc3fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'abd62b4b16da4dcd8e0bdbcb6d9db3a8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:57.265741 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d15b822d-f384-46d1-9c9d-0f37b11bc3fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'abd62b4b16da4dcd8e0bdbcb6d9db3a8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:57.266404 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d15b822d-f384-46d1-9c9d-0f37b11bc3fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'abd62b4b16da4dcd8e0bdbcb6d9db3a8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:57.274377 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-d15b822d-f384-46d1-9c9d-0f37b11bc3fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'abd62b4b16da4dcd8e0bdbcb6d9db3a8'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:57.281194 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1437/2887] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:19:57 2022] DELETE /identity/v3/users/abd62b4b16da4dcd8e0bdbcb6d9db3a8 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:19:57.600366 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42e406-8de8-49af-a3ed-3b435353476a service placement] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:19:57.611536 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac42e406-8de8-49af-a3ed-3b435353476a service placement] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:19:57.680289 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb330d3c-ca72-4384-82a9-a21198f48c44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:19:57.681178 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb330d3c-ca72-4384-82a9-a21198f48c44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:19:57.681466 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb330d3c-ca72-4384-82a9-a21198f48c44 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:19:57.681748 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb330d3c-ca72-4384-82a9-a21198f48c44 admin admin] PATH_INFO: `/v3/projects/7528adefacfc41c5a54869ec5603bde8` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:19:57.682960 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb330d3c-ca72-4384-82a9-a21198f48c44 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7528adefacfc41c5a54869ec5603bde8)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:19:57.685037 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb330d3c-ca72-4384-82a9-a21198f48c44 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:19:57.735504 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-eb330d3c-ca72-4384-82a9-a21198f48c44 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7528adefacfc41c5a54869ec5603bde8', 'request_id': 'req-eb330d3c-ca72-4384-82a9-a21198f48c44'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:57.741388 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-eb330d3c-ca72-4384-82a9-a21198f48c44 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7528adefacfc41c5a54869ec5603bde8', 'request_id': 'req-eb330d3c-ca72-4384-82a9-a21198f48c44'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:57.749791 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-eb330d3c-ca72-4384-82a9-a21198f48c44 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7528adefacfc41c5a54869ec5603bde8', 'request_id': 'req-eb330d3c-ca72-4384-82a9-a21198f48c44'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:57.751025 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:19:57.764083 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-eb330d3c-ca72-4384-82a9-a21198f48c44 admin admin] The token cache is being invalidate because project 7528adefacfc41c5a54869ec5603bde8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:19:57.764381 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-eb330d3c-ca72-4384-82a9-a21198f48c44 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:57.765030 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-eb330d3c-ca72-4384-82a9-a21198f48c44 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:57.766904 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1451/2888] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:19:57 2022] DELETE /identity/v3/projects/7528adefacfc41c5a54869ec5603bde8 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:19:57.770666 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b822d-f384-46d1-9c9d-0f37b11bc3fe admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:19:57.772100 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-d15b822d-f384-46d1-9c9d-0f37b11bc3fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:19:57.785432 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d15b822d-f384-46d1-9c9d-0f37b11bc3fe admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:19:57.844101 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9166e-2d27-4f9e-806d-bf1af43e2cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:19:57.844925 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9166e-2d27-4f9e-806d-bf1af43e2cf6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:19:57.845218 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9166e-2d27-4f9e-806d-bf1af43e2cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:19:57.845503 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e9166e-2d27-4f9e-806d-bf1af43e2cf6 admin admin] PATH_INFO: `/v3/users/61667a8c1e2c461d8f9891b71ea7a2e4` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:19:57.846709 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9166e-2d27-4f9e-806d-bf1af43e2cf6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=61667a8c1e2c461d8f9891b71ea7a2e4)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:19:57.848688 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e9166e-2d27-4f9e-806d-bf1af43e2cf6 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:19:57.895594 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-17e9166e-2d27-4f9e-806d-bf1af43e2cf6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61667a8c1e2c461d8f9891b71ea7a2e4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:57.908981 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-17e9166e-2d27-4f9e-806d-bf1af43e2cf6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61667a8c1e2c461d8f9891b71ea7a2e4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:57.918935 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-17e9166e-2d27-4f9e-806d-bf1af43e2cf6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61667a8c1e2c461d8f9891b71ea7a2e4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:57.919719 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-17e9166e-2d27-4f9e-806d-bf1af43e2cf6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61667a8c1e2c461d8f9891b71ea7a2e4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:57.928184 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-17e9166e-2d27-4f9e-806d-bf1af43e2cf6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61667a8c1e2c461d8f9891b71ea7a2e4'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:57.934844 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1438/2889] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:19:57 2022] DELETE /identity/v3/users/61667a8c1e2c461d8f9891b71ea7a2e4 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:19:58.108284 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb330d3c-ca72-4384-82a9-a21198f48c44 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:19:58.109825 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-eb330d3c-ca72-4384-82a9-a21198f48c44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:19:58.123560 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb330d3c-ca72-4384-82a9-a21198f48c44 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:19:58.185271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55d7279b-4b98-47e1-8719-02cd8d5afb54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:19:58.186135 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7279b-4b98-47e1-8719-02cd8d5afb54 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:19:58.186418 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7279b-4b98-47e1-8719-02cd8d5afb54 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:19:58.186693 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55d7279b-4b98-47e1-8719-02cd8d5afb54 admin admin] PATH_INFO: `/v3/projects/99ed0f4b2e7e4ab1a79c30dd247c8061` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:19:58.187910 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d7279b-4b98-47e1-8719-02cd8d5afb54 admin admin] RBAC: Authorizing `identity:delete_project(project_id=99ed0f4b2e7e4ab1a79c30dd247c8061)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:19:58.189824 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55d7279b-4b98-47e1-8719-02cd8d5afb54 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:19:58.238392 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-55d7279b-4b98-47e1-8719-02cd8d5afb54 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '99ed0f4b2e7e4ab1a79c30dd247c8061', 'request_id': 'req-55d7279b-4b98-47e1-8719-02cd8d5afb54'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:58.243778 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-55d7279b-4b98-47e1-8719-02cd8d5afb54 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '99ed0f4b2e7e4ab1a79c30dd247c8061', 'request_id': 'req-55d7279b-4b98-47e1-8719-02cd8d5afb54'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:58.252299 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-55d7279b-4b98-47e1-8719-02cd8d5afb54 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '99ed0f4b2e7e4ab1a79c30dd247c8061', 'request_id': 'req-55d7279b-4b98-47e1-8719-02cd8d5afb54'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:58.263743 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-55d7279b-4b98-47e1-8719-02cd8d5afb54 admin admin] The token cache is being invalidate because project 99ed0f4b2e7e4ab1a79c30dd247c8061 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:19:58.264075 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-55d7279b-4b98-47e1-8719-02cd8d5afb54 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:58.264800 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-55d7279b-4b98-47e1-8719-02cd8d5afb54 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:19:58.266433 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1452/2890] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:19:58 2022] DELETE /identity/v3/projects/99ed0f4b2e7e4ab1a79c30dd247c8061 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:20:07.931446 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=76,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:08.260776 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:20:59.710198 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9166e-2d27-4f9e-806d-bf1af43e2cf6 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:20:59.711978 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-17e9166e-2d27-4f9e-806d-bf1af43e2cf6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:59.810543 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e9166e-2d27-4f9e-806d-bf1af43e2cf6 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:20:59.874511 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e3ce7-fa2f-4d8e-ab0b-724f8a1fcb9c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b738f1b8c64590955a640c461b4ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:20:59.875184 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e3ce7-fa2f-4d8e-ab0b-724f8a1fcb9c service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:20:59.875474 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e3ce7-fa2f-4d8e-ab0b-724f8a1fcb9c service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:20:59.875774 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-673e3ce7-fa2f-4d8e-ab0b-724f8a1fcb9c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:20:59.877917 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-673e3ce7-fa2f-4d8e-ab0b-724f8a1fcb9c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:20:59.889700 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e3ce7-fa2f-4d8e-ab0b-724f8a1fcb9c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:20:59.891297 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-673e3ce7-fa2f-4d8e-ab0b-724f8a1fcb9c service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:20:59.901436 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1439/2891] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:20:59 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:21:09.897330 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=67 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:21:24.619186 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155058b-d779-47c2-ad2f-20dc881d2e7f None None] REQUEST_METHOD: `POST` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:21:24.619769 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155058b-d779-47c2-ad2f-20dc881d2e7f None None] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:21:24.619769 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f155058b-d779-47c2-ad2f-20dc881d2e7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:21:24.639925 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.auth.core [None req-f155058b-d779-47c2-ad2f-20dc881d2e7f None None] MFA Rules not processed for user `67b7bca2e2ef4dbf89554ba2f9bf93e5`. Rule list: `[]` (Enabled: `True`). {{(pid=80232) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} May 25 16:21:24.641208 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f155058b-d779-47c2-ad2f-20dc881d2e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:21:24.699601 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1453/2892] 10.222.0.85 () {58 vars in 957 bytes} [Wed May 25 16:21:24 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) May 25 16:21:34.692362 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=25 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:23:20.409176 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e3ce7-fa2f-4d8e-ab0b-724f8a1fcb9c service placement] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:20.410947 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-673e3ce7-fa2f-4d8e-ab0b-724f8a1fcb9c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:20.473310 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-673e3ce7-fa2f-4d8e-ab0b-724f8a1fcb9c service placement] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:20.477678 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773e0d8-4b6c-49c8-bf72-110184d7cfab service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:20.478357 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773e0d8-4b6c-49c8-bf72-110184d7cfab service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:20.478650 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773e0d8-4b6c-49c8-bf72-110184d7cfab service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:20.478934 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7773e0d8-4b6c-49c8-bf72-110184d7cfab service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:20.481042 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-7773e0d8-4b6c-49c8-bf72-110184d7cfab service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:20.492637 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773e0d8-4b6c-49c8-bf72-110184d7cfab service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:20.494109 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7773e0d8-4b6c-49c8-bf72-110184d7cfab service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:20.503909 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1440/2893] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:23:20 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:23:21.361100 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155058b-d779-47c2-ad2f-20dc881d2e7f None None] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:21.363445 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-f155058b-d779-47c2-ad2f-20dc881d2e7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:21.479644 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f155058b-d779-47c2-ad2f-20dc881d2e7f None None] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:21.486513 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8eed9-4984-4e5b-b9b1-6fa2a625bfde service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:21.487180 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8eed9-4984-4e5b-b9b1-6fa2a625bfde service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:21.487912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8eed9-4984-4e5b-b9b1-6fa2a625bfde service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:21.487912 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb8eed9-4984-4e5b-b9b1-6fa2a625bfde service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:21.489673 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0fb8eed9-4984-4e5b-b9b1-6fa2a625bfde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:21.502936 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8eed9-4984-4e5b-b9b1-6fa2a625bfde service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:21.504667 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb8eed9-4984-4e5b-b9b1-6fa2a625bfde service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:21.515759 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1454/2894] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:23:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:23:21.534139 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773e0d8-4b6c-49c8-bf72-110184d7cfab service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:21.544092 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7773e0d8-4b6c-49c8-bf72-110184d7cfab service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:21.548566 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d02ad0-d42a-4a7e-b028-94242138ad65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:21.549303 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d02ad0-d42a-4a7e-b028-94242138ad65 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:21.550094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d02ad0-d42a-4a7e-b028-94242138ad65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:21.550094 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d02ad0-d42a-4a7e-b028-94242138ad65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:21.552003 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c1d02ad0-d42a-4a7e-b028-94242138ad65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:21.568145 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d02ad0-d42a-4a7e-b028-94242138ad65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:21.569811 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1d02ad0-d42a-4a7e-b028-94242138ad65 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:21.581717 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1441/2895] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:23:21 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:23:22.578428 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8eed9-4984-4e5b-b9b1-6fa2a625bfde service nova] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:22.580248 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0fb8eed9-4984-4e5b-b9b1-6fa2a625bfde service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:22.593684 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb8eed9-4984-4e5b-b9b1-6fa2a625bfde service nova] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:22.598082 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa9275-c899-4f3a-8e4a-215f36379ecd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:22.598735 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa9275-c899-4f3a-8e4a-215f36379ecd service neutron] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:22.599022 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa9275-c899-4f3a-8e4a-215f36379ecd service neutron] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:22.599315 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0caa9275-c899-4f3a-8e4a-215f36379ecd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:22.601271 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0caa9275-c899-4f3a-8e4a-215f36379ecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:22.611202 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa9275-c899-4f3a-8e4a-215f36379ecd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:22.612746 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0caa9275-c899-4f3a-8e4a-215f36379ecd service neutron] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:22.621049 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1455/2896] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:23:22 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:23:22.791280 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d02ad0-d42a-4a7e-b028-94242138ad65 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:22.800423 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d02ad0-d42a-4a7e-b028-94242138ad65 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:22.804258 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f264732-2a2e-4410-87b6-1d5db7fe2ca8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:22.804906 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f264732-2a2e-4410-87b6-1d5db7fe2ca8 service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:22.805195 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f264732-2a2e-4410-87b6-1d5db7fe2ca8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:22.805475 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f264732-2a2e-4410-87b6-1d5db7fe2ca8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:22.807356 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4f264732-2a2e-4410-87b6-1d5db7fe2ca8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:22.822190 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f264732-2a2e-4410-87b6-1d5db7fe2ca8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:22.823652 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f264732-2a2e-4410-87b6-1d5db7fe2ca8 service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:22.835140 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1442/2897] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:23:22 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:23:30.473391 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa9275-c899-4f3a-8e4a-215f36379ecd service neutron] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:30.475158 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-0caa9275-c899-4f3a-8e4a-215f36379ecd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:30.492278 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0caa9275-c899-4f3a-8e4a-215f36379ecd service neutron] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:30.499298 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eda8ed-ad52-44c4-ac54-c2bfda211ea2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '4751abc9f67947b3ad17ba2b78ce2c12', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['ResellerAdmin', 'admin', 'reader', 'service', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:30.500021 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eda8ed-ad52-44c4-ac54-c2bfda211ea2 service nova] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:30.500299 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eda8ed-ad52-44c4-ac54-c2bfda211ea2 service nova] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:30.500585 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59eda8ed-ad52-44c4-ac54-c2bfda211ea2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:30.502820 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-59eda8ed-ad52-44c4-ac54-c2bfda211ea2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:30.516357 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eda8ed-ad52-44c4-ac54-c2bfda211ea2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:30.517942 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59eda8ed-ad52-44c4-ac54-c2bfda211ea2 service nova] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:30.528692 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1456/2898] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:23:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:23:32.830290 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=38 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:23:33.879966 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f264732-2a2e-4410-87b6-1d5db7fe2ca8 service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:33.889316 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f264732-2a2e-4410-87b6-1d5db7fe2ca8 service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:33.893172 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c173cf0-8c76-4a0f-bdf0-1e862406f65b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '413b3d65427b41dc99bc7b0a94fb1dc8', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:33.893839 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c173cf0-8c76-4a0f-bdf0-1e862406f65b service neutron] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:33.894132 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c173cf0-8c76-4a0f-bdf0-1e862406f65b service neutron] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:33.894418 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c173cf0-8c76-4a0f-bdf0-1e862406f65b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:33.896261 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4c173cf0-8c76-4a0f-bdf0-1e862406f65b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:33.965168 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c173cf0-8c76-4a0f-bdf0-1e862406f65b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:33.967164 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c173cf0-8c76-4a0f-bdf0-1e862406f65b service neutron] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:33.977535 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1443/2899] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:23:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:23:37.345167 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eda8ed-ad52-44c4-ac54-c2bfda211ea2 service nova] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:37.346820 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-59eda8ed-ad52-44c4-ac54-c2bfda211ea2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:37.361834 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59eda8ed-ad52-44c4-ac54-c2bfda211ea2 service nova] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:37.368680 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1604c-9419-41c8-84cf-bbe969d21152 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b738f1b8c64590955a640c461b4ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:37.369406 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1604c-9419-41c8-84cf-bbe969d21152 service placement] REQUEST_METHOD: `GET` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:37.369857 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1604c-9419-41c8-84cf-bbe969d21152 service placement] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:37.370138 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8d1604c-9419-41c8-84cf-bbe969d21152 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:37.372118 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-d8d1604c-9419-41c8-84cf-bbe969d21152 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:37.385287 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1604c-9419-41c8-84cf-bbe969d21152 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:37.386957 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8d1604c-9419-41c8-84cf-bbe969d21152 service placement] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:37.399543 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1457/2900] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:23:37 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:23:40.352780 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c173cf0-8c76-4a0f-bdf0-1e862406f65b service neutron] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:40.355012 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-4c173cf0-8c76-4a0f-bdf0-1e862406f65b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:40.370612 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c173cf0-8c76-4a0f-bdf0-1e862406f65b service neutron] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:40.375581 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b21c83-b227-4643-87c6-bd19eae2a7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:40.376593 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b21c83-b227-4643-87c6-bd19eae2a7fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:40.376886 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b21c83-b227-4643-87c6-bd19eae2a7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:40.377201 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b21c83-b227-4643-87c6-bd19eae2a7fb admin admin] PATH_INFO: `/v3/users/67b7bca2e2ef4dbf89554ba2f9bf93e5` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:40.378452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b21c83-b227-4643-87c6-bd19eae2a7fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=67b7bca2e2ef4dbf89554ba2f9bf93e5)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:40.380564 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b21c83-b227-4643-87c6-bd19eae2a7fb admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:40.437147 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b3b21c83-b227-4643-87c6-bd19eae2a7fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '67b7bca2e2ef4dbf89554ba2f9bf93e5'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:40.449163 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b3b21c83-b227-4643-87c6-bd19eae2a7fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '67b7bca2e2ef4dbf89554ba2f9bf93e5'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:40.456993 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b3b21c83-b227-4643-87c6-bd19eae2a7fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '67b7bca2e2ef4dbf89554ba2f9bf93e5'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:40.457794 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b3b21c83-b227-4643-87c6-bd19eae2a7fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '67b7bca2e2ef4dbf89554ba2f9bf93e5'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:40.465005 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-b3b21c83-b227-4643-87c6-bd19eae2a7fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '67b7bca2e2ef4dbf89554ba2f9bf93e5'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:40.472655 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1444/2901] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:23:40 2022] DELETE /identity/v3/users/67b7bca2e2ef4dbf89554ba2f9bf93e5 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) May 25 16:23:40.664859 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1604c-9419-41c8-84cf-bbe969d21152 service placement] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:40.677284 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8d1604c-9419-41c8-84cf-bbe969d21152 service placement] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:40.734716 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b2af8-2410-433c-a955-cf9b5502b047 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:40.735593 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b2af8-2410-433c-a955-cf9b5502b047 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:40.735918 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b2af8-2410-433c-a955-cf9b5502b047 admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:40.736200 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4b2af8-2410-433c-a955-cf9b5502b047 admin admin] PATH_INFO: `/v3/projects/31462b97f7a04859b66398fd6386cfc4` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:40.737410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b2af8-2410-433c-a955-cf9b5502b047 admin admin] RBAC: Authorizing `identity:delete_project(project_id=31462b97f7a04859b66398fd6386cfc4)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:40.739342 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4b2af8-2410-433c-a955-cf9b5502b047 admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:40.762313 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:SELECT=100 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:23:40.786324 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ba4b2af8-2410-433c-a955-cf9b5502b047 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '31462b97f7a04859b66398fd6386cfc4', 'request_id': 'req-ba4b2af8-2410-433c-a955-cf9b5502b047'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:40.791719 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ba4b2af8-2410-433c-a955-cf9b5502b047 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '31462b97f7a04859b66398fd6386cfc4', 'request_id': 'req-ba4b2af8-2410-433c-a955-cf9b5502b047'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:40.800180 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ba4b2af8-2410-433c-a955-cf9b5502b047 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '31462b97f7a04859b66398fd6386cfc4', 'request_id': 'req-ba4b2af8-2410-433c-a955-cf9b5502b047'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:40.812120 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ba4b2af8-2410-433c-a955-cf9b5502b047 admin admin] The token cache is being invalidate because project 31462b97f7a04859b66398fd6386cfc4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:23:40.812410 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ba4b2af8-2410-433c-a955-cf9b5502b047 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:40.813444 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-ba4b2af8-2410-433c-a955-cf9b5502b047 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:40.815110 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1458/2902] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:23:40 2022] DELETE /identity/v3/projects/31462b97f7a04859b66398fd6386cfc4 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:23:40.819223 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b21c83-b227-4643-87c6-bd19eae2a7fb admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:40.820738 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-b3b21c83-b227-4643-87c6-bd19eae2a7fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:40.834688 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b21c83-b227-4643-87c6-bd19eae2a7fb admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:40.893452 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c10ff-f25e-4f99-a101-d7a28973c1b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:40.894368 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c10ff-f25e-4f99-a101-d7a28973c1b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:40.894641 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c10ff-f25e-4f99-a101-d7a28973c1b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:40.894923 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0c10ff-f25e-4f99-a101-d7a28973c1b4 admin admin] PATH_INFO: `/v3/users/92a57e680b63468c8ba0c0f976434168` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:40.896158 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c10ff-f25e-4f99-a101-d7a28973c1b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=92a57e680b63468c8ba0c0f976434168)` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:40.898122 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0c10ff-f25e-4f99-a101-d7a28973c1b4 admin admin] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:40.903603 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:23:40.949932 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb0c10ff-f25e-4f99-a101-d7a28973c1b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '92a57e680b63468c8ba0c0f976434168'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:40.962276 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb0c10ff-f25e-4f99-a101-d7a28973c1b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '92a57e680b63468c8ba0c0f976434168'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:40.971066 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb0c10ff-f25e-4f99-a101-d7a28973c1b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '92a57e680b63468c8ba0c0f976434168'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:40.971722 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb0c10ff-f25e-4f99-a101-d7a28973c1b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '92a57e680b63468c8ba0c0f976434168'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:40.978716 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.notifications [None req-cb0c10ff-f25e-4f99-a101-d7a28973c1b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '92a57e680b63468c8ba0c0f976434168'} {{(pid=80233) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:40.984630 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1445/2903] 10.222.0.85 () {58 vars in 1223 bytes} [Wed May 25 16:23:40 2022] DELETE /identity/v3/users/92a57e680b63468c8ba0c0f976434168 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:23:41.162104 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b2af8-2410-433c-a955-cf9b5502b047 admin admin] Authenticating user token {{(pid=80232) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:23:41.163630 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.fernet_utils [None req-ba4b2af8-2410-433c-a955-cf9b5502b047 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80232) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:23:41.184527 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4b2af8-2410-433c-a955-cf9b5502b047 admin admin] Validating token access rules against request {{(pid=80232) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:23:41.242151 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f22c256-7da8-4d1c-90e5-6258d612b88d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7effefc613a44b799e8f81fc6a35b678', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '12da861032034c60b092c703b1fe5e90', 'project_domain_id': 'default', 'roles': ['member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80232) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:23:41.243036 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22c256-7da8-4d1c-90e5-6258d612b88d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:23:41.243348 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22c256-7da8-4d1c-90e5-6258d612b88d admin admin] SCRIPT_NAME: `/identity` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:23:41.243625 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f22c256-7da8-4d1c-90e5-6258d612b88d admin admin] PATH_INFO: `/v3/projects/125dc8f9e58b436a9103f12b6968a2cd` {{(pid=80232) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:23:41.244884 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22c256-7da8-4d1c-90e5-6258d612b88d admin admin] RBAC: Authorizing `identity:delete_project(project_id=125dc8f9e58b436a9103f12b6968a2cd)` {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:23:41.246808 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f22c256-7da8-4d1c-90e5-6258d612b88d admin admin] RBAC: Authorization granted {{(pid=80232) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:23:41.292911 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-4f22c256-7da8-4d1c-90e5-6258d612b88d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '125dc8f9e58b436a9103f12b6968a2cd', 'request_id': 'req-4f22c256-7da8-4d1c-90e5-6258d612b88d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:41.298677 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-4f22c256-7da8-4d1c-90e5-6258d612b88d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '125dc8f9e58b436a9103f12b6968a2cd', 'request_id': 'req-4f22c256-7da8-4d1c-90e5-6258d612b88d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:41.307879 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-4f22c256-7da8-4d1c-90e5-6258d612b88d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '125dc8f9e58b436a9103f12b6968a2cd', 'request_id': 'req-4f22c256-7da8-4d1c-90e5-6258d612b88d'} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:41.320728 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-4f22c256-7da8-4d1c-90e5-6258d612b88d admin admin] The token cache is being invalidate because project 125dc8f9e58b436a9103f12b6968a2cd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80232) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} May 25 16:23:41.321016 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-4f22c256-7da8-4d1c-90e5-6258d612b88d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:41.321727 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG keystone.notifications [None req-4f22c256-7da8-4d1c-90e5-6258d612b88d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80232) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} May 25 16:23:41.323499 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: [pid: 80232|app: 0|req: 1459/2904] 10.222.0.85 () {58 vars in 1232 bytes} [Wed May 25 16:23:41 2022] DELETE /identity/v3/projects/125dc8f9e58b436a9103f12b6968a2cd => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) May 25 16:23:50.981574 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:23:51.317380 dv-cinder-843277-2-iscsi devstack@keystone.service[80232]: DEBUG dbcounter [-] [80232] Writing DB stats keystone:DELETE=14,keystone:SELECT=66,keystone:INSERT=2 {{(pid=80232) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} May 25 16:26:08.532056 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c10ff-f25e-4f99-a101-d7a28973c1b4 admin admin] Authenticating user token {{(pid=80233) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} May 25 16:26:08.533751 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-cb0c10ff-f25e-4f99-a101-d7a28973c1b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:08.618155 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0c10ff-f25e-4f99-a101-d7a28973c1b4 admin admin] Validating token access rules against request {{(pid=80233) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} May 25 16:26:08.685753 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d834fd-c07e-4926-aa56-c21ffeba7a03 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '82b738f1b8c64590955a640c461b4ef9', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5584ebdaf133444ca69efc5554bf48e9', 'project_domain_id': 'default', 'roles': ['service', 'member', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80233) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} May 25 16:26:08.686421 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d834fd-c07e-4926-aa56-c21ffeba7a03 service placement] REQUEST_METHOD: `GET` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} May 25 16:26:08.686709 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d834fd-c07e-4926-aa56-c21ffeba7a03 service placement] SCRIPT_NAME: `/identity` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} May 25 16:26:08.686999 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d834fd-c07e-4926-aa56-c21ffeba7a03 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80233) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} May 25 16:26:08.689204 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.fernet_utils [None req-c8d834fd-c07e-4926-aa56-c21ffeba7a03 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80233) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} May 25 16:26:08.701269 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d834fd-c07e-4926-aa56-c21ffeba7a03 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} May 25 16:26:08.702896 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d834fd-c07e-4926-aa56-c21ffeba7a03 service placement] RBAC: Authorization granted {{(pid=80233) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} May 25 16:26:08.713778 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: [pid: 80233|app: 0|req: 1446/2905] 10.222.0.85 () {60 vars in 1365 bytes} [Wed May 25 16:26:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 182 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) May 25 16:26:18.709156 dv-cinder-843277-2-iscsi devstack@keystone.service[80233]: DEBUG dbcounter [-] [80233] Writing DB stats keystone:SELECT=58 {{(pid=80233) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}